-- Logs begin at Sun 2022-08-28 09:35:56 UTC, end at Sun 2022-08-28 10:37:56 UTC. -- Aug 28 10:01:44.507654 dv-cinder-852009-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 28 10:01:44.518259 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 28 10:01:44 2022] *** Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: nodename: dv-cinder-852009-4-iscsi Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: machine: x86_64 Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: clock source: unix Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: pcre jit disabled Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: detected number of CPU cores: 4 Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: current working directory: / Aug 28 10:01:44.520321 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: detected binary path: /usr/bin/uwsgi-core Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: your processes number limit is 47680 Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: your memory page size is 4096 bytes Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: detected max file descriptor number: 2048 Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: lock engine: pthread robust mutexes Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: thunder lock: enabled Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 28 10:01:44.521005 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 28 10:01:44.560189 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: Python main interpreter initialized at 0x55df60f349c0 Aug 28 10:01:44.560189 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: python threads support enabled Aug 28 10:01:44.560189 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: your server socket listen backlog is limited to 100 connections Aug 28 10:01:44.560189 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: your mercy for graceful operations on workers is 90 seconds Aug 28 10:01:44.560599 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: mapped 403077 bytes (393 KB) for 2 cores Aug 28 10:01:44.560681 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: *** Operational MODE: preforking *** Aug 28 10:01:44.560929 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: *** uWSGI is running in multiple interpreter mode *** Aug 28 10:01:44.560929 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: spawned uWSGI master process (pid: 81683) Aug 28 10:01:44.561141 dv-cinder-852009-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 28 10:01:44.563044 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 1 (pid: 81697, cores: 1) Aug 28 10:01:44.563044 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 2 (pid: 81698, cores: 1) Aug 28 10:01:44.563044 dv-cinder-852009-4-iscsi devstack@keystone.service[81683]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 28 10:01:48.792832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 10:01:48.792832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Aug 28 10:01:48.793666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 10:01:48.793666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Aug 28 10:01:48.794077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 10:01:48.794077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Aug 28 10:01:48.794184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 10:01:48.794184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Aug 28 10:01:48.842277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 10:01:48.842448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 10:01:48.842524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 10:01:48.842602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] command line args: [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 10:01:48.842667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 10:01:48.842805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 10:01:48.842895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] command line args: [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 10:01:48.842961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ================================================================================ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 10:01:48.843096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 10:01:48.843230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ================================================================================ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 10:01:48.843469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_dir = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_dir = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_source = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.843951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] debug = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_source = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] debug = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.844913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_config_append = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_config_append = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.845911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_dir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_dir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.846915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.847967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.848911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.849881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] publish_errors = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] publish_errors = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.850995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] standard_threads = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] standard_threads = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.851937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_journal = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.852925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_journal = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_json = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_json = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_stderr = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_stderr = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_syslog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_syslog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.853909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.854036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 10:01:48.854161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.854946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.855928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.856743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.857946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.858971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.859966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.860905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.861960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.862998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.863901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.864870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.865803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.866916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.867940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.868967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.869900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.870944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.871887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872139 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.872950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.873983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.874948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.875910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.876971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.877898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.878803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.879927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.880878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.881932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.882972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.external = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.external = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.883912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.password = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.password = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.token = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.token = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.884997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.885890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.886922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.887951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.888929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.889967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.890997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.891906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.892969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.893987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.894974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.895893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.896886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.897927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.898963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.899995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.900934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.901956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.902993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.903886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.904982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.905999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.906959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.907859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.908926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.909951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.910980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.911892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.912879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.913912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.914950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.915980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.916902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.917911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.918959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.919084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.919209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.919335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.919879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.920954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.921977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.922759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.923893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.924878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.925965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.926930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.927934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.928920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.929877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.930967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.931931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.932972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.933926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.934752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.935985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.936961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.937919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.938870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 10:01:48.939966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 10:01:48.939966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 10:01:48.955761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.955886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.956022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.956112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.956246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.956337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.964747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.964844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.992723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.992867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.992936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:48.992998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.045752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.046182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.093926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.094057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.094423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.094659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.250815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.251960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.274720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.274852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.274921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.275940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.276065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.276148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.287079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.287205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.300943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.301790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 28 10:01:49.318677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.318931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 28 10:01:49.319055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.319125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 28 10:01:49.319186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.319411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 28 10:01:49.319535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.319604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 28 10:01:49.319664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.319897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.320780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.330933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.331060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.331216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.338048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.338319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.341082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.341281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.341407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.341573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.341718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.341834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.341936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.342949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.343872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.344075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.366980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.367242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.368442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.368751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.368877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.369039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.373467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.373593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.373731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.373930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.374071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.374207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.380440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.380748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.380871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.381099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.385029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.385226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.385326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.385505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.385631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.385779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.385866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.386915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.388262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.388552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.388868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.404283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.404580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.404705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.404955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.408463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.408862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.411753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.412021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.412192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.412346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.416401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.416818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.419938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.420330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.423248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.423545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.423669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.423800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.423883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.424717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.425050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.440824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.441112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.441242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.441322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.441514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.441658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.441794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.441990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.448656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.448944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.450201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.450622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.451734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.452007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.452131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.452438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.454697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.454983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.455107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.455349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.455445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.455671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.455798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.455936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.456029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.456231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.467771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.467898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.468036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.468993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.469072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.469279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.469405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.469543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.469624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.485309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.485454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.488240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.488368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.491213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.491341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.492692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.492819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.494300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.494426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.494564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.494642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.494845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.494971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.501379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.501506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.502869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.503016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.505881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.506934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.514783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.514920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.515096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.515213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.515302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.515420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.522378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.522526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 28 10:01:49.522663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.522797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.522876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.523839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.524933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.525890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.526010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 28 10:01:49.560530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55df60f349c0 pid: 81697 (default app) Aug 28 10:01:49.560774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55df60f349c0 pid: 81698 (default app) Aug 28 10:01:49.563831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70ed1-ff25-46d5-b609-3ae4a7f50759 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:49.564223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70ed1-ff25-46d5-b609-3ae4a7f50759 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:49.564325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70ed1-ff25-46d5-b609-3ae4a7f50759 None None] PATH_INFO: `/v3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:49.568632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Sun Aug 28 10:01:49 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:01:56.127452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:56.128009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:56.128009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:56.132461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:01:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:01:56.141530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cd304-768b-42ae-a090-391d650975e9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:56.141999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cd304-768b-42ae-a090-391d650975e9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:56.142137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cd304-768b-42ae-a090-391d650975e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:56.220137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: INFO dbcounter [None req-667cd304-768b-42ae-a090-391d650975e9 None None] Registered counter for database keystone Aug 28 10:01:56.241259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG oslo_db.sqlalchemy.engines [None req-667cd304-768b-42ae-a090-391d650975e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81697) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 10:01:56.253172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writer thread running {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 10:01:56.438960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-667cd304-768b-42ae-a090-391d650975e9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81697) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 28 10:01:56.439308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-667cd304-768b-42ae-a090-391d650975e9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81697) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 28 10:01:56.494498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-667cd304-768b-42ae-a090-391d650975e9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:01:56.496104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-667cd304-768b-42ae-a090-391d650975e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:01:56.642425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:01:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 28 10:01:57.237064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:01:57.312186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: INFO dbcounter [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] Registered counter for database keystone Aug 28 10:01:57.336165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG oslo_db.sqlalchemy.engines [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81698) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 10:01:57.347071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writer thread running {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 10:01:57.471359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caff02-3ef7-4d74-84ba-648a4454d5c5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:01:57.476927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:01:57.477544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:57.477821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:57.478087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:57.487145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:01:57.538501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING keystone.common.rbac_enforcer.enforcer [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 28 10:01:57.538837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db021b-83b5-4346-adee-6f17351b2445 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:01:57.549316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Aug 28 10:01:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:01:57.556565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cd304-768b-42ae-a090-391d650975e9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:01:57.575806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cd304-768b-42ae-a090-391d650975e9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:01:57.581558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:01:57.582701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:57.582981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:57.583313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] PATH_INFO: `/v3/projects/371955716dbe4a589ec9dad82a9828ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:57.585518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] RBAC: Authorizing `identity:get_project(project_id=371955716dbe4a589ec9dad82a9828ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:01:57.637589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 28 10:01:57.637924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28882912-1bd3-48cf-9409-b1d78005a19e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:01:57.642598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Sun Aug 28 10:01:57 2022] GET /identity/v3/projects/371955716dbe4a589ec9dad82a9828ca => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:01:59.505006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6cbb-b98e-4817-8b43-481bc4a1a04e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:59.505613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6cbb-b98e-4817-8b43-481bc4a1a04e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:59.505613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6cbb-b98e-4817-8b43-481bc4a1a04e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:59.509799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:01:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:01:59.516470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:59.517211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:59.517357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:59.540192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:01:59.541743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:01:59.551924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:01:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 28 10:01:59.834554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6cbb-b98e-4817-8b43-481bc4a1a04e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:01:59.853429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6cbb-b98e-4817-8b43-481bc4a1a04e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:01:59.858673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:01:59.859274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:59.859551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:59.859819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:59.868049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:01:59.869868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4832dcb-1ad8-407e-b24c-a6f72195bf56 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:01:59.878702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Aug 28 10:01:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:01:59.885229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:01:59.900587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a762fa3-b4ae-4f39-89be-53b37b1f8400 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:01:59.909991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:01:59.910961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:01:59.911388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:01:59.911817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:01:59.913393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:01:59.915228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789c20-0d47-46f5-b5ba-5587fac9ac2f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:01:59.916778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Sun Aug 28 10:01:59 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:01.863238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9f96-c593-426c-a91b-813591c17abc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:01.863761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9f96-c593-426c-a91b-813591c17abc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:01.863761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f9f96-c593-426c-a91b-813591c17abc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:01.865352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:01.870844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:01.871282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:01.871373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:01.890468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:01.891650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:01.915429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc8e5-7fc5-477e-83e6-c3cdeb0d9de6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:01.915716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc8e5-7fc5-477e-83e6-c3cdeb0d9de6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:01.915992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc8e5-7fc5-477e-83e6-c3cdeb0d9de6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:01.917344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:01.923173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:01.923458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:01.923806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:01.967868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81698) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 28 10:02:01.968259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81698) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 28 10:02:01.977528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:02.013324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:02.014935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:02.118196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:02.237990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.262573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.341016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969329e-ab5b-41a1-b813-0817217a7b03 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.375049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.375620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.375905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.376144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.379431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.387992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:02.397080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:02.397970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:02.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:02.403977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.482749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.484656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae297-7699-4d6c-919b-89498a12cc2b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.487028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.500785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c9f72-17fa-4639-a7b9-8408ad98ef7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.523380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.524321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.524699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.524980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.533774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.534373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.534650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.534919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.535589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:02.535891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.537351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.549088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:02.554813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.594483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.596391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.656986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc321fd-c515-4be6-8404-ab3cf12a0900 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.690976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.691612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.691884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.692150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.693361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.694210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:02.694840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.696404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:02.699642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.701752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.703150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.716375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.801763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61794f8-f84d-4a0c-bbf1-12a316f3cb2e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.805730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a31b-b382-4131-8009-6365bfb4aa80 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.834028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.834878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.835154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.835423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.840186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.840828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.841104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.841374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.842539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.844031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.845703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:02.851711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.853885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.855378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:02:02.874621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:02.875946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:02:02.876761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:02:02.876761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:02:02.876761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:02.876761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:02.881784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:02.932486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.941545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:02.953084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb808730-78eb-4d2f-b32e-6e8bede50532 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.985822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c51-4b72-49ce-ab2a-108d2a900beb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:02.987564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:02.988207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:02.988530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:02.988825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:02.989555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.domain.id=default, user.id=9a1bcfcdf46a4813b52d81367ec40dec) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:02.989918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:02.991379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07bc7-213d-4716-8486-a384dacb58cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:02.997979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/role_assignments?user.id=9a1bcfcdf46a4813b52d81367ec40dec&scope.domain.id=default&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:03.020316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:03.020984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:03.021294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:03.021562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:03.022232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:03.022531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:03.023964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddd50-9d1b-450e-b3f3-e9a86111a48e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:03.043256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:05.078117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063afc7-8b9b-4913-8b63-9570ee4941b2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.079239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063afc7-8b9b-4913-8b63-9570ee4941b2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.079737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063afc7-8b9b-4913-8b63-9570ee4941b2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.081649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:05.086956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c93568-4c19-4c7f-abf8-f53555ca0460 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.087790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c93568-4c19-4c7f-abf8-f53555ca0460 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.088278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c93568-4c19-4c7f-abf8-f53555ca0460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.113497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d0c93568-4c19-4c7f-abf8-f53555ca0460 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:05.115101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d0c93568-4c19-4c7f-abf8-f53555ca0460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:05.200277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 11/23] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:05.323478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e5a9-7f78-4da3-95c6-9afa3cd58b29 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.324208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e5a9-7f78-4da3-95c6-9afa3cd58b29 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.324717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e5a9-7f78-4da3-95c6-9afa3cd58b29 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.326267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 13/24] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:05.331600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b076-cd50-4f3f-8535-e3fe388eca71 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.332121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b076-cd50-4f3f-8535-e3fe388eca71 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.332635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b076-cd50-4f3f-8535-e3fe388eca71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.361547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b071b076-cd50-4f3f-8535-e3fe388eca71 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:05.362756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b071b076-cd50-4f3f-8535-e3fe388eca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:05.414099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:05.446162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 12/25] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:05.594120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a543e64-8e4c-4acb-a521-95242a30249a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.594690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a543e64-8e4c-4acb-a521-95242a30249a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.595171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a543e64-8e4c-4acb-a521-95242a30249a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.596753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 14/26] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:05.602202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.602742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.603212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.622785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a543e64-8e4c-4acb-a521-95242a30249a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:05.642285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:05.645751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:05.795126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:05.801878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a543e64-8e4c-4acb-a521-95242a30249a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:05.812994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:05.887616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:05.888270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.888707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.888879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.916632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a65df23b-f123-43bc-a922-a9eae3b245e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:05.922398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:05.923226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:05.923957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:05.930590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 15/28] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:05.933762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea027-030e-4786-916e-dd61e5c8962b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:05.934042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea027-030e-4786-916e-dd61e5c8962b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:05.934315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea027-030e-4786-916e-dd61e5c8962b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:05.935897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 16/29] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:05.937625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea027-030e-4786-916e-dd61e5c8962b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:05.948929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.032128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f557c3-39ee-4cfe-adb9-8d532ae07254 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.044468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.093674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.094349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.094593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.094874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.112249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.113813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.118038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea027-030e-4786-916e-dd61e5c8962b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c810179b-d2a8-48de-93ab-fb50bee66b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:02:06.129188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:06.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:02:06.130859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:02:06.130859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:02:06.130859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:06.130859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 14/30] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:06.131488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.133509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.133509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.153611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.154297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.154636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.155012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.155749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:06.156173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.157429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:06.157928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.158625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:06.167545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 17/31] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:06.173391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.232224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 15/32] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:06.232927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.254936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.299172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3ca03-9031-4245-a190-02dca77f4b41 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.327270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.337014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.338018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.338492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.338997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.339873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.340260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4a04-1511-48bf-9c15-ada68fb9f11c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.341852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.377367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.378018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.378298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.378567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.379272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:06.379786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.381332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.394893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 16/33] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:06.395647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.485156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 18/34] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Aug 28 10:02:06 2022] POST /identity/v3/roles => generated 219 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:06.486493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.512018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178622-014e-4379-94c7-9b30f121ab24 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.556708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.557741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.558233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.558695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.560879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.563272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.565462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 17/35] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:06 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:06.572387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.596363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.640203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240d81-55a1-470d-9eb9-bcabac7a5307 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.668468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.678193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.678872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.679153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.679426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.681348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.682988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.692344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2420-da0c-42a7-b53d-3fff1ef20141 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.721973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 19/36] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:02:06 2022] POST /identity/v3/projects => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:06.738664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.739833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.740369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.740981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.742592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.744553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.746674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 18/37] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:06.753382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.770339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:06.894911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:06.916413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70175c-fc3f-4925-b236-50fb8d27855b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:06.965561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:06.966636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:06.968336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:06.968849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:06.969768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:06.972985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70c22-0b08-445d-b037-d7d7f6761723 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:06.989633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a27d5-e485-4e93-8be4-369e22a71acc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:07.043180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:07.044699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.045194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.045679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] PATH_INFO: `/v3/domains/default/users/9a1bcfcdf46a4813b52d81367ec40dec/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.047866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9a1bcfcdf46a4813b52d81367ec40dec, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:07.049599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266f8f-85d8-4dc1-9151-2f8df0841fab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:07.070100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 20/38] 10.222.0.40 () {58 vars in 1361 bytes} [Sun Aug 28 10:02:06 2022] PUT /identity/v3/domains/default/users/9a1bcfcdf46a4813b52d81367ec40dec/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:07.127960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 19/39] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Aug 28 10:02:06 2022] POST /identity/v3/roles => generated 225 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:07.139456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461446-c64d-444b-98cb-d856b718bdb3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.139994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461446-c64d-444b-98cb-d856b718bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.139994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2461446-c64d-444b-98cb-d856b718bdb3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.141555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 21/40] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:07.148269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.149018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.149510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.172094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e26d6-76f3-427a-bb2a-d2b766debc72 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.172458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e26d6-76f3-427a-bb2a-d2b766debc72 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.172695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e26d6-76f3-427a-bb2a-d2b766debc72 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.174093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 22/41] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:07.176016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:07.178249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:07.180205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.180496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.180800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.204101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:07.205890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:07.245502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:07.284194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 20/42] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:07.297988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 23/43] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:07.547380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:07.558560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:07.631641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:07.659510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdb919-0827-4a3e-83be-8e001ebc1952 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:07.666662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b5263-a5e8-4a84-923a-9d09cf5ecc6b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:07.692096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:07.692768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.693089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.693352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.694527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:07.696127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:07.697759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 21/44] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:07.703521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:07.703840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:07.704462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.704823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.705041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.705758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:07.707147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99bfdd-8296-48bf-9e0a-a9d6fb738837 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:07.722518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 24/45] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Aug 28 10:02:07 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:02:07.821857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63651a5-6f18-4046-9030-e4a7f7583687 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:07.865421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:07.866049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:07.866302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:07.866573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:07.867257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:07.868728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862c05d-3535-4f8b-851f-2057265fc868 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:07.993827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:08.008488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 22/46] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:02:07 2022] POST /identity/v3/projects => generated 312 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:09.029464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f9cab-8ec0-465e-a464-622d3f5d8af7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.030004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f9cab-8ec0-465e-a464-622d3f5d8af7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.030004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f9cab-8ec0-465e-a464-622d3f5d8af7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.031369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 25/47] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:09.036771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.037174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.037367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.057412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:09.058685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:09.131109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 23/48] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:09.392041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f9cab-8ec0-465e-a464-622d3f5d8af7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:09.425248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:09.615993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f9cab-8ec0-465e-a464-622d3f5d8af7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:09.651751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:09.652306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.652618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.652887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.663567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:09.668482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:09.669701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:09.670652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 26/49] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:09.675977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:09.778100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90b99-c783-4cc3-9c91-455a39c593fa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:09.801946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:09.813768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:09.814390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.814671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.814944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.815619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:09.815918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:09.817338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85509d8-6576-423c-acb0-38401c582f26 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:09.829188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 24/50] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:09.835464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:09.878839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8626e-e219-4017-b965-89f507e9bc33 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.879125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8626e-e219-4017-b965-89f507e9bc33 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.879403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8626e-e219-4017-b965-89f507e9bc33 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.880949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 25/51] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:09.885995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.886283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.886554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.904911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:09.906067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:09.929631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:09.941726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6463f0a-b4c6-4b58-bd42-5dcb3dd7c54a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:09.976020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:09.976676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:09.976956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:09.977204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 26/52] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:09.977302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:09.978365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:09.979818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:09.981407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:09 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:09.986968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:10.086521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869efe-730a-48cd-abc6-b34c4e601fd9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:10.103810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:10.122370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:10.122999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:10.123281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:10.123551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:10.124716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:10.126181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:10.127762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 27/54] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:10.133544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:10.235558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605bf5e-3b94-4c89-ae77-a5ae2fb0a99e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:10.241139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:10.270218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:10.270854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:10.271160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:10.271427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:10.272160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.domain.id=default, user.id=9a1bcfcdf46a4813b52d81367ec40dec) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:10.272475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:10.273936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:10.279594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 28/55] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Aug 28 10:02:10 2022] GET /identity/v3/role_assignments?user.id=9a1bcfcdf46a4813b52d81367ec40dec&scope.domain.id=default&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:10.348224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc870ce-ba89-49f2-ba24-d91d550d257a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:10.387694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:10.388347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:10.388666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:10.388938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:10.390154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:10.391633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27c6c1-86be-454b-ab07-204c9644d72b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:10.393322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 28/56] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:10.399203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:10.423452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:10.507215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b63a8-23c5-43c6-afa5-70492b00d76d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:10.540838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:10.541448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:10.541731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:10.542003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:10.542658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:10.544491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fa3bc-5bb4-4246-9006-d656585b5ea8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:10.573491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 29/57] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Aug 28 10:02:10 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:12.452087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0e3c-8b15-46a6-8085-bdbf70487a94 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:12.452661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0e3c-8b15-46a6-8085-bdbf70487a94 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:12.452661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0e3c-8b15-46a6-8085-bdbf70487a94 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:12.454058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 29/58] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:12.460477 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:12.461278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:12.461416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:12.484135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:12.485358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:12.553710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 30/59] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:12.818968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f0e3c-8b15-46a6-8085-bdbf70487a94 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:12.879507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:12.929703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f0e3c-8b15-46a6-8085-bdbf70487a94 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:12.965636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:12.966253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:12.966530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:12.966808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:12.968048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:12.969539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a4e4a-5543-4b4b-b33b-e8e472fc7631 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:12.971106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 30/60] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:12.976986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:12.979328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:13.088931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48a48e-fc91-4a4e-b7f1-1c6d4812b680 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:13.132772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:13.133688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:13.134103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:13.134498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:13.135510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:13.137947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dab3bf-838f-4bb5-bd5a-66f2b05fe8ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:13.171189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 31/61] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:12 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:16.110323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ac90-515b-4bd6-8e9f-bf430a3f15c3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.111176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ac90-515b-4bd6-8e9f-bf430a3f15c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.111590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ac90-515b-4bd6-8e9f-bf430a3f15c3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.113162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 31/62] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:16.118595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39497f1-005c-4d2a-ba65-deedfc07f0ea None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.119389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39497f1-005c-4d2a-ba65-deedfc07f0ea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.119858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39497f1-005c-4d2a-ba65-deedfc07f0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.138196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a39497f1-005c-4d2a-ba65-deedfc07f0ea None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:16.139704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a39497f1-005c-4d2a-ba65-deedfc07f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:16.176098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c68ec-6562-4ef6-977a-e01d613f685e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.176723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c68ec-6562-4ef6-977a-e01d613f685e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.177301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c68ec-6562-4ef6-977a-e01d613f685e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.178877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 32/63] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:16.188832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e0d9-2b4d-4ee7-8974-0a3ed557d7ee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.189117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e0d9-2b4d-4ee7-8974-0a3ed557d7ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.189411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e0d9-2b4d-4ee7-8974-0a3ed557d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.215380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8743e0d9-2b4d-4ee7-8974-0a3ed557d7ee None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:16.217447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8743e0d9-2b4d-4ee7-8974-0a3ed557d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:16.220757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:16.243892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 32/64] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:16.312249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 33/65] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:16.497783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0839d-9123-4ab1-add5-a88ed6e9f501 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.498015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0839d-9123-4ab1-add5-a88ed6e9f501 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.498370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0839d-9123-4ab1-add5-a88ed6e9f501 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.500096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 33/66] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:16.505264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c05fe-82e1-4b50-9ae6-4d30f275196e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.505556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c05fe-82e1-4b50-9ae6-4d30f275196e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.505835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c05fe-82e1-4b50-9ae6-4d30f275196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.525432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e18c05fe-82e1-4b50-9ae6-4d30f275196e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:16.526641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e18c05fe-82e1-4b50-9ae6-4d30f275196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:16.576386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0839d-9123-4ab1-add5-a88ed6e9f501 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:16.590181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:16.626756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 34/67] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:16.696018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1297-243f-4185-9fd8-1a2a115463d4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.696706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1297-243f-4185-9fd8-1a2a115463d4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.697174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1297-243f-4185-9fd8-1a2a115463d4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.698948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 35/68] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:16.699876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1297-243f-4185-9fd8-1a2a115463d4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:16.731664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0839d-9123-4ab1-add5-a88ed6e9f501 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:16.775283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:16.776531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.777113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.777726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.791718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b8930a4c-34a4-4a10-9f23-108595d85edd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:16.798324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:16.799125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:16.799864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:16.802102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 34/69] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:16.807108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.807108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.807108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.817940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1297-243f-4185-9fd8-1a2a115463d4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:16.828875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:16.830877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:16.853933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:16.854577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:16.854862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:16.855138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:16.867330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:16.874812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:16.875678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:16.876396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 36/70] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:16.877078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:16.893221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:16.911832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 35/71] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:16.912795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:16.937465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.001935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071e80-39db-4111-b6fd-4bd2be11c3b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.021029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f58af-e352-413f-ab85-04be6fa8b14b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.037680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.038478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.038762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.039033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.039703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:17.040004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.041428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.054224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.054224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.061737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.062658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.062954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.063237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.063922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:17.064234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.065694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24720b0-b538-429a-a928-bb8ac3c34f1c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 36/73] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4283b9-4f52-4f43-9465-981e6426556c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4283b9-4f52-4f43-9465-981e6426556c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4283b9-4f52-4f43-9465-981e6426556c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 37/74] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:17.079245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4283b9-4f52-4f43-9465-981e6426556c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.125737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.161217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325953d-bdcb-4032-a87b-cda7d8612956 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.162774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.190681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4283b9-4f52-4f43-9465-981e6426556c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.195730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.196372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.196680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.196950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.207281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:17.212312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:17.213271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:17.213997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 38/75] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:17.214822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.229747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.230396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.230681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.230955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.232103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.233628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca65aa-7f4f-43d6-b753-fea881486ae4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.235228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 38/76] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.236323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.236637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.236914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.254039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:17.255299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:17.318922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134b7b-abf4-4143-9921-872aad777c03 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.320056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 39/77] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:17.320918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.353425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.354063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.354349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.354629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.370536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.372019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.373716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 39/78] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.374428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.374556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.401994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.439704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2923a-2cd0-4c2a-8f64-6cd0e62aeef4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.479333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.480465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.481107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.481469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.482711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172b59-b2aa-4959-8475-0861c070b516 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.494586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:17.501214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:17.502230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:17.503154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:17.503875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 40/79] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:17.508605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.518001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.518774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.519159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.519530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.520311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:17.520784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.522284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.532214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 40/80] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.537298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.618307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.621681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.622550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb12c96-c61f-4e45-aa67-bfd65b75df61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.640300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f9ca8-9393-4bf3-99b9-91b7c90cfcbe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.656884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.657503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.657785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.658056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.673803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.674407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.674684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.674952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.722743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.724287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.741332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.742807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.817753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.821739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:17.824658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 41/81] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:17.825539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.845209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 41/82] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:17.845209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.925043 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09a9a4-09e3-4961-9208-eb16a9d95a35 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.937544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0333d-256a-4a1d-b949-7007b61a6c26 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:17.958782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.959383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.959659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.959925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.960627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:17.960928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.962357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.970715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 42/83] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.971493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:17.973632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:17.974338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:17.974621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:17.974893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:17.976093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:17.977587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:17.983729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 42/84] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:17.983729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.030861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.033120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.072343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df530a9c-a211-4323-869b-126f8de328d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.082531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2072a-0e9d-44eb-a316-aeead2f68ec5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.105204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.105767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.106051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.106324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.107491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.109013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.110581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 43/85] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.111367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.116471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.117148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.117438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.117716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.118848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.120317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.121974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 43/86] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:17 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.122751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.212569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728d4e8-9686-42e8-a7ad-12ba6987fcf3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.218789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b7a0-f563-440e-be63-8d246de00cb7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.235106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.240010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.248521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.249157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.249435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.249711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.250822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.252286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.253326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.253947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 44/87] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.254060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.254225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.254500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.254698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.255745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.257227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.259068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 44/88] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.259807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.359625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e969596-5e37-49b4-9cb8-d7a29fa07983 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.359853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ef8-a3b8-4733-b8bc-d749be69807d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.394192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.394433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.394807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.394897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.395088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.395173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.395357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.395449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.396108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.397605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.426278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 45/89] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:18 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:18.427192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.475508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.477754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.484635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.486185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.548098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a36e-9ea2-4f72-bee7-882680eaf37c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.592550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.592550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.592902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.593117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.593867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=9a1bcfcdf46a4813b52d81367ec40dec) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:18.594175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.595656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.601016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 45/90] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:18.601879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.610208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 46/91] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/role_assignments?user.id=9a1bcfcdf46a4813b52d81367ec40dec&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.610879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.735598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.743817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a8af5-d640-4a45-a728-aeb8d46aeccb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.764335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89ca5-c702-4e1e-a500-3bb3a1dc0281 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.773236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:18.795175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.795796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.796075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.796926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.797223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:18.797533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.799016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.805542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 46/92] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.806368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:18.821236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.822519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.823005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.823471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.913075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.915328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.928934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf062c5e-ffa3-481b-9f8f-9241e72d35ac None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:18.967091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:18.968172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:18.968688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:18.969177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:18.970642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:18.972409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:18.974667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 47/93] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:18.982794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:19.036620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:19.102706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 47/94] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:19.110085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:19.122006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:19.168651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13c274-e24e-4d7c-9fd4-d10fb7e1db3c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:19.206992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:19.207660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:19.207944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:19.208305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:19.209080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:19.209440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:19.210918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:19.217242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 48/95] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:18 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=07556e60b8b243f0844a294e71af2a62 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:19.248054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df65b-f90e-403a-90f7-ec165a76650f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:19.304094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:19.304803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:19.305084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:19.305364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] PATH_INFO: `/v3/projects/73d4df0979174d19936f9bf141b3b93d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:19.306538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] RBAC: Authorizing `identity:get_project(project_id=73d4df0979174d19936f9bf141b3b93d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:19.308017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e5491-4797-4f44-8239-d890b225e642 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:19.310101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:19 2022] GET /identity/v3/projects/73d4df0979174d19936f9bf141b3b93d => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:19.315848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:19.454075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:19.463661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e07967-79de-497d-987e-a4a388f9e67a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:19.509634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:19.510527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:19.510942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:19.511346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:19.512231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=73d4df0979174d19936f9bf141b3b93d, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:19.512709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:19.514350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ead7d-84e3-45a7-98a5-b5bb2a477ed4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:19.521303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:19 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=73d4df0979174d19936f9bf141b3b93d&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:21.352619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fdf1-fb16-49f3-a31c-88eeb65a04df None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.353178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fdf1-fb16-49f3-a31c-88eeb65a04df None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.353178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073fdf1-fb16-49f3-a31c-88eeb65a04df None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.354664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:21.359677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffb138-1b4f-4640-ac70-9712445b20bd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.360062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffb138-1b4f-4640-ac70-9712445b20bd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.360236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffb138-1b4f-4640-ac70-9712445b20bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.388446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1ffb138-1b4f-4640-ac70-9712445b20bd None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:21.389669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1ffb138-1b4f-4640-ac70-9712445b20bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:21.483189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:21.727764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a417f9-e6c1-45fe-b028-4f7a20887e7b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.728139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a417f9-e6c1-45fe-b028-4f7a20887e7b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.728317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a417f9-e6c1-45fe-b028-4f7a20887e7b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.729858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 50/100] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:21.735105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdf39-9d3a-4de5-81b2-8776947be0e2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.735396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdf39-9d3a-4de5-81b2-8776947be0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.735674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdf39-9d3a-4de5-81b2-8776947be0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.760754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-503fdf39-9d3a-4de5-81b2-8776947be0e2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:21.761959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-503fdf39-9d3a-4de5-81b2-8776947be0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:21.822447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0b8d-51ef-448d-964f-b8b0d5b445e9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.822777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0b8d-51ef-448d-964f-b8b0d5b445e9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.823005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0b8d-51ef-448d-964f-b8b0d5b445e9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.824526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 51/101] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:21.829854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.830151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.830427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.853777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:21.854974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:21.872529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 51/102] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:21.874080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd61b3-6cf7-4716-9c71-d81fea301398 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.874504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd61b3-6cf7-4716-9c71-d81fea301398 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.874943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd61b3-6cf7-4716-9c71-d81fea301398 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.876759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 52/103] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:21.885807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:21.886266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:21.886679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:21.914220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:21.915759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:21.935652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:21.982949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 52/104] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:21.984017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:21.992051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.041029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 53/105] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:22.112865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4f5b1-6b98-43b9-9a97-573fcea1e34a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.153087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.153728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.154009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.154283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.173385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:22.178354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:22.179168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:22.179888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 53/106] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:22.185442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.283075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.337616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0aa497-186e-44f3-9baa-16b9c5b3c302 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.392464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.393111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.393378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.393652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.394337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:22.394642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:22.396083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bef58-a003-4f60-bf62-ee26bfc1ce61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:22.405723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.406190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 54/107] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:22.407143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95479a-a895-4cac-82b0-4fd5f646331c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.407428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95479a-a895-4cac-82b0-4fd5f646331c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.407703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95479a-a895-4cac-82b0-4fd5f646331c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.409258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 55/108] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:22.413294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95479a-a895-4cac-82b0-4fd5f646331c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.425011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.452805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b58a4-e573-4219-9746-f3b80fb3cb32 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.485905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.486541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.486817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.487084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.496874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:22.501678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:22.502519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 54/109] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:22.508469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.516674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95479a-a895-4cac-82b0-4fd5f646331c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.550714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.551277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.551552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.551973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.553070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:22.554533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf486-387b-44f3-8fee-0311cf0ace47 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:22.556106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 56/110] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:22.557505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5840-ca5f-4ea4-8419-5e7edfbe672d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.557786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5840-ca5f-4ea4-8419-5e7edfbe672d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.558054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5840-ca5f-4ea4-8419-5e7edfbe672d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.560621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 57/111] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:22.562094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb5840-ca5f-4ea4-8419-5e7edfbe672d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.605389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcbe82-e0ec-40f7-8fe4-ee7c71c28775 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.628489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.639848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.640491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.640804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.641079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.642195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.651506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9877d2b8-3f9d-4838-a523-316f00e5aefc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:22.656564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:22.657425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:22.658154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 55/112] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:22.659014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.659260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.659519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.668866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb5840-ca5f-4ea4-8419-5e7edfbe672d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.677075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:22.678264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:22.701599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.702239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.702519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.702790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] PATH_INFO: `/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.715706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] RBAC: Authorizing `identity:get_user(user_id=fa7bbedcf83448bc9417ba8cd9fcca8b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:22.717310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:22.718931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 58/113] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:22.721634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.745167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 56/114] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:22.745999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:22.841574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4253945-d72f-4cda-8da7-9f43c06ae5ac None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.847022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f4928-f612-4544-8394-ada3414be97f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:22.861818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.877803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.878421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.878710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.878990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.879662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:22.880002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:22.881444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8723fbc-b51f-4c35-849a-6744fac12558 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:22.882907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:22.889473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:22.890322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.890464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 59/115] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:22.895463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:22.895463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.895463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.895685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:22.895685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:22.896175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:22.897763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:22.909985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:22.911224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:22.928317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 57/116] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:02:22 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:22.938459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.009232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 60/117] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:23.010426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.063092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572dc6-55c4-4708-a4d1-30e21286cf1e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.105755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.106742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.107747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.108216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.109581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:23.110090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.112182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.121109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 58/118] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.122212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.143441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6eae4-df41-42f5-9902-6587f8e00c77 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.163301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.179552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.188624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.189322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.189739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.190101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.241985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc27b0d-ec68-4b1f-8466-6331dd32e849 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.260487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.262154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.287222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.288099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.288784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.289068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.290728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.292947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.296059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 59/119] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.297024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.367431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 61/120] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:23.368285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.395089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.414749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.421759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c810fc-ea8f-4624-80a9-0edd4a55d53a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.457536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.458505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.458786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.459059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.469116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:23.474502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:23.475333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:23.476062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 60/121] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:23.476698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.487867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcf1e5-c182-4570-92f6-c07e8bb5a1be None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.524854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.525487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.525771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.526046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.527185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.528698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.530294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 62/122] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.535460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.589758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267e63-3b4a-474f-9964-316aa2c0deca None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.624070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.624670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.624956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.625228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.635848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.637147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:23.643091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:23.643929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:23.644704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:23.645265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35cde-8c77-4332-8608-2919fe66bd95 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.645340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 61/123] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:23.645340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.666980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.685190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.686594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.686898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.687173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.688303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.689812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.696764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 63/124] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.696764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.749304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c147-5467-4771-8b5a-9d263d2c8ed7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.784968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.785611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.786294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.786592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.788041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.789721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.796471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 62/125] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.796471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.814431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4053ce5-90b8-4057-be05-74719be0944d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.849254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.849896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.850175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.850425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.851097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:23.851391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.852792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.861039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 64/126] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.866247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.885654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.892405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:23.903036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f360d3-036e-49ff-a5e8-f8a3a13f2746 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:23.939397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:23.940045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:23.940323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:23.940620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:23.941739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:23.943251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:23.945039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 63/127] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:23.949801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:23.976342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9b4e-279f-4f5f-bb65-41f5e57844f9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.010354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.010975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.011257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.011562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.012206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:24.012547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.013949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.022331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 65/128] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.022951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.070206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3418e-7bf0-4d08-945f-83c4e85e5089 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.105177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.105853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.106138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.106411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/9a1bcfcdf46a4813b52d81367ec40dec/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.108336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, user_id=9a1bcfcdf46a4813b52d81367ec40dec, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.109841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.121900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:24.126058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 64/129] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:23 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/9a1bcfcdf46a4813b52d81367ec40dec/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:24.126874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.143399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:24.156553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6a1d-b61b-4f11-9248-722b83ce94d9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.202992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.203919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.203919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.204268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.205064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=fa7bbedcf83448bc9417ba8cd9fcca8b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:24.205413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.206989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.213672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:24 2022] GET /identity/v3/role_assignments?user.id=fa7bbedcf83448bc9417ba8cd9fcca8b&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.214619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.257687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926271-eb0b-48e0-8e02-07369ff54b41 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.306004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.307107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.307640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.308131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.309805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.312420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.315618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:24 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.316792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.374616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003419d0-2054-45b4-9699-cdb57e1933d2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.424391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.425379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.425674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.425950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.428672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, user_id=b2c1bfd1c0e7420b9a8651065c3d719d, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.428904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:24.430165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.449777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d2aa-0023-4d06-a053-aec403d22ce2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.451076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 67/132] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:24 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:24.452658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.471817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:24.494125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.495214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.495713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.496226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.497785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.499507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.501510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 66/133] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:24 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.509825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.583131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906c3ac-9b37-4166-852d-a78617530f82 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.622982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.623644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.623926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.624197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.626158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.627868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.630102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 68/134] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:24 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.637899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.766233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:24.806485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554bc99-6f22-4e51-983b-654c08294d23 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.808764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12525e26-c719-4ce6-b667-937ce8d137e5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:24.855429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.856121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.856405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.856731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.859368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, user_id=b2c1bfd1c0e7420b9a8651065c3d719d, role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.861264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.887325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 69/135] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:24 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/07556e60b8b243f0844a294e71af2a62 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:24.892170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:24.892983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.893327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.893603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] PATH_INFO: `/v3/projects/73d4df0979174d19936f9bf141b3b93d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.894872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] RBAC: Authorizing `identity:get_project(project_id=73d4df0979174d19936f9bf141b3b93d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:24.896521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa2b73-695a-4560-ab10-06d6d37bcd7f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:24.898363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 67/136] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:24 2022] GET /identity/v3/projects/73d4df0979174d19936f9bf141b3b93d => generated 312 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:24.905212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:24.971989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af0b2-4b55-4bbb-b994-d3362824feb5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.972317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af0b2-4b55-4bbb-b994-d3362824feb5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.972600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857af0b2-4b55-4bbb-b994-d3362824feb5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:24.974021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 68/137] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:24.979246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:24.979537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:24.979826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:25.000692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:25.001912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:25.016408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:25.024009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b87b85-3d3f-4d1b-a8f8-12ba34b84155 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:25.076666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:25.077459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:25.077842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:25.078128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] PATH_INFO: `/v3/projects/73d4df0979174d19936f9bf141b3b93d/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:25.080300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] RBAC: Authorizing `identity:create_grant(project_id=73d4df0979174d19936f9bf141b3b93d, user_id=b2c1bfd1c0e7420b9a8651065c3d719d, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:25.081989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:25.082252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 69/138] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:25.112312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 70/139] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:24 2022] PUT /identity/v3/projects/73d4df0979174d19936f9bf141b3b93d/users/b2c1bfd1c0e7420b9a8651065c3d719d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:25.113087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:25.374498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:25.478562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d574b2-bc88-4bee-a520-55d2512cfdbe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:25.514370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:25.515014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:25.515296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:25.515568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:25.516725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:25.518182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272181e-e717-4725-b2ab-eadfcbb2c32e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:25.520082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 70/140] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:25.525663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:25.634237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89593c5d-9c26-45f9-b352-c667a2a6714a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:25.668671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:25.669630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:25.669909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:25.670190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:25.670866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:25.672341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b9b0e-87c2-4980-a122-8c2fad6e6de7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:25.710816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 71/141] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:25 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:26.771642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cae92-c394-4379-98f6-00e57298b897 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:26.772149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cae92-c394-4379-98f6-00e57298b897 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:26.772222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cae92-c394-4379-98f6-00e57298b897 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:26.773623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 71/142] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:26.778905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df499-e6c3-4859-9b3b-500bbc2dd1d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:26.779315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df499-e6c3-4859-9b3b-500bbc2dd1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:26.779483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df499-e6c3-4859-9b3b-500bbc2dd1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:26.798571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-150df499-e6c3-4859-9b3b-500bbc2dd1d6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:26.799762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-150df499-e6c3-4859-9b3b-500bbc2dd1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:26.856541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:26.877723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 72/143] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:26.980565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6807-2465-4934-ac32-8b80507c9da8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:26.980879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6807-2465-4934-ac32-8b80507c9da8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:26.981126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6807-2465-4934-ac32-8b80507c9da8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:26.982725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 72/144] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:26.988287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3e917-c278-417f-a802-31ceb9076f2f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:26.988607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3e917-c278-417f-a802-31ceb9076f2f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:26.988880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3e917-c278-417f-a802-31ceb9076f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.009762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5cf3e917-c278-417f-a802-31ceb9076f2f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:27.010953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cf3e917-c278-417f-a802-31ceb9076f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:27.096928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 73/145] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:27.466625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d6807-2465-4934-ac32-8b80507c9da8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:27.483304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:27.495532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598305-1c28-414b-a4bc-0f4baee8ba31 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.495863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598305-1c28-414b-a4bc-0f4baee8ba31 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.496038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598305-1c28-414b-a4bc-0f4baee8ba31 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.497429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 74/146] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:27.501922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335a0f-63ce-4f33-bda3-5163c1b6cd60 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.502201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335a0f-63ce-4f33-bda3-5163c1b6cd60 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.502500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39335a0f-63ce-4f33-bda3-5163c1b6cd60 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.503776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 75/147] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:27.504928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406e6c8-e3ef-4534-9bac-9ec66135453c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.505282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406e6c8-e3ef-4534-9bac-9ec66135453c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.505474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406e6c8-e3ef-4534-9bac-9ec66135453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.537014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6406e6c8-e3ef-4534-9bac-9ec66135453c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:27.538227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6406e6c8-e3ef-4534-9bac-9ec66135453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:27.630781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 76/148] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:27.632086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.632367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.632672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.637744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d6807-2465-4934-ac32-8b80507c9da8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:27.660887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:27.662176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:27.719543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:27.720157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.720443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.720743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.737713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:27.749593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:27.755505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:27.756326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:27.757106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:27.760844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 73/149] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:27.762014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:27.806555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 77/150] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:27.807523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:27.941944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095d59-f116-439b-ada0-ad0431cebb95 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:27.954424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:27.996325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:27.996895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:27.997173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:27.997441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:27.998116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:27.998413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:27.999824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.013186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 78/151] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:28.021395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.023987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc24724-fffb-4688-bce4-b83cadf1b999 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.070476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:28.093237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.093891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.094172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.094444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.113905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:28.123610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:28.124461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:28.125234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 74/152] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:28.149570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.165522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2260-6122-4434-993e-882e54c569b1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.208670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.209322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.209598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.209867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] PATH_INFO: `/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.215255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] RBAC: Authorizing `identity:get_user(user_id=fa7bbedcf83448bc9417ba8cd9fcca8b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:28.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.222121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 79/153] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:28.228430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.328377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797fe3-33bb-4283-9dda-84a52aa1eb20 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.375439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.376062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.376341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.376636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.377319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:28.377622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:28.379027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.386323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:28.403014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 75/154] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:28.403769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.451675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aa096-e8e1-436f-9385-5b461c072f6c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.456319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:28.514804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.515870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.516472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.516971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.518315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:28.520184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.528567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd9d1-cb35-4e53-86a6-f0b68be1754b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.529682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 80/155] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:28.531087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.561576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.562218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.562511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.562767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.575626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:28.581584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:28.582429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:28.583483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 76/156] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:28.584558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.651362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:28.698405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f2328-4c93-4e18-a0fb-d0994693c81a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.734979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.735791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.736170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.736562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] PATH_INFO: `/v3/users/9a1bcfcdf46a4813b52d81367ec40dec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.737903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9a1bcfcdf46a4813b52d81367ec40dec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:28.740004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.742108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 77/157] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/users/9a1bcfcdf46a4813b52d81367ec40dec => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:28.743376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.766724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829988a0-f194-47c0-85a7-6d63ef0b98a8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.825819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.826701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.826867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.827020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.844563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c36e4-cc9f-457f-8649-8fbfbc08a2f5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:28.849653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:28.860982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:28.863067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 81/158] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:28.865166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.874885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:28.879590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:28.880311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:28.880669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:28.880944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/fa7bbedcf83448bc9417ba8cd9fcca8b/roles/9b8364ce17a6400b8f88cfbeee0a13f0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:28.883157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, user_id=fa7bbedcf83448bc9417ba8cd9fcca8b, role_id=9b8364ce17a6400b8f88cfbeee0a13f0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:28.884368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:28.914625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 78/159] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:28 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/users/fa7bbedcf83448bc9417ba8cd9fcca8b/roles/9b8364ce17a6400b8f88cfbeee0a13f0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:28.915477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:28.939114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:29.131512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395aaa5-5e5a-487d-bcad-2cc5418edd9f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.137418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61961c-9dbf-463b-8050-680991dd4e3d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.174789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.175415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.175691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.175960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.176694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:29.177000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.178460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.188257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 82/160] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:29.189129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:29.204534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.205333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.205615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.205977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.302594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:29.331625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.334048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.338836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:29.351508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2532-4d6e-493b-bbe4-d2dfc9672b48 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.419500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.420046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.420324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.420624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.421311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:29.421610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.423045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496031aa-2e58-480b-bbf1-4e3edb90b6aa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.442680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 83/161] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:29.444192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc304c59-ac85-4070-8f5a-3762755fa594 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.444468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc304c59-ac85-4070-8f5a-3762755fa594 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.444765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc304c59-ac85-4070-8f5a-3762755fa594 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.448230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 84/162] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:29.449028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc304c59-ac85-4070-8f5a-3762755fa594 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:29.535067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 79/163] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:29.540162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:29.633346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc304c59-ac85-4070-8f5a-3762755fa594 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.681669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.682226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.682502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.682764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.683898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.685842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d54452-fda5-4764-bfda-f77c018ccf70 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.687615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 85/164] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:29.692295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac596d9-5263-48ef-be76-f3486972d443 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.692612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac596d9-5263-48ef-be76-f3486972d443 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.692907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac596d9-5263-48ef-be76-f3486972d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.725576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4ac596d9-5263-48ef-be76-f3486972d443 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:29.726762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4ac596d9-5263-48ef-be76-f3486972d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:29.733043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eedd2f-eef8-4263-a945-a62840f1e26d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.745187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:29.770573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.771700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.771978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.772250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.773448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.775023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.776851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 80/165] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:29.777609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:29.789995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:29.923801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 86/166] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:29.924926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34c6-8725-47b1-98bc-c37e98af61f4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.925207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34c6-8725-47b1-98bc-c37e98af61f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.925475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34c6-8725-47b1-98bc-c37e98af61f4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.926905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 87/167] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:29.927594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f34c6-8725-47b1-98bc-c37e98af61f4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:29.936622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788de2db-e6bc-466c-8e27-1ee570215dbf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:29.971177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:29.972198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:29.972729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:29.973198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:29.974880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:29.976598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:29.978932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 81/168] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:29.979978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.034091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.133883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f34c6-8725-47b1-98bc-c37e98af61f4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.162433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fd595-80d4-4771-825e-69a52c5a942c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.197320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.197863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.198138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.198410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.226981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.227606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.227881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.228150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.243470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.295115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:30.296627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3f267-453b-4802-ac91-621035ddb8e6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:30.316532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:30.318095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:30.344235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.402803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 88/169] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:30.404061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eefe2-3783-4af3-bdf0-5e4f1d65e9f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.404341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eefe2-3783-4af3-bdf0-5e4f1d65e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.404657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eefe2-3783-4af3-bdf0-5e4f1d65e9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.422947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e6eefe2-3783-4af3-bdf0-5e4f1d65e9f8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:30.424127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e6eefe2-3783-4af3-bdf0-5e4f1d65e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:30.450950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 82/170] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:30.451954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.479606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.498027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 89/171] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:30.499085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81d93-1b5a-4984-b2b7-fedae33a551b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.499369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81d93-1b5a-4984-b2b7-fedae33a551b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.499636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81d93-1b5a-4984-b2b7-fedae33a551b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.501045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 90/172] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:30.501597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b81d93-1b5a-4984-b2b7-fedae33a551b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.580708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ddf91-172f-42cd-bf0c-73b68aee2618 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.602940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.617339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.618179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.618285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.618489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.619238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=9a1bcfcdf46a4813b52d81367ec40dec) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:30.619556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:30.621313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:30.626189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b81d93-1b5a-4984-b2b7-fedae33a551b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.627316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 83/173] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/role_assignments?user.id=9a1bcfcdf46a4813b52d81367ec40dec&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:30.628173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.667415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.668464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.669396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.669913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.683075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:30.689102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:30.690031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:30.690827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:30.693515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 91/174] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:30.694808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.750471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.811066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49392955-3efe-4704-b8af-e5743a0d4b99 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.825814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d6b-45d0-4753-adbc-8e9c0266c2c6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:30.848801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.850320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.851087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.851566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.853603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:30.856336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bf982-20db-49c2-9732-8af108ba0053 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:30.858672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 92/175] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:30.860374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.860916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.861337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.864600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:30.865237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.865523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.865793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.866993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:30.870373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e8cc1-a510-4401-acaf-59098e194fa2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:30.872080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 84/176] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:30.873441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c750d4-9cd3-4509-8962-e3749c85c9b0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:30.873723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c750d4-9cd3-4509-8962-e3749c85c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:30.873992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c750d4-9cd3-4509-8962-e3749c85c9b0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:30.875332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 85/177] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:30.875936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c750d4-9cd3-4509-8962-e3749c85c9b0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:30.884300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:30.885813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:30.946891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:30.970286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 93/178] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:30.971243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.009667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c750d4-9cd3-4509-8962-e3749c85c9b0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.020379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.052187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.052837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.053118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.053390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.054425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:31.054560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.055850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.065097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 86/179] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.065882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.080605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715984e-e0ab-473d-82e9-7de0ca3849c6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.115614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.116615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.117089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.117541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.128711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:31.133908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:31.134728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:31.135490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:31.137819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 94/180] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:31.138863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.175113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31350751-146c-49b1-a289-48170afe4db2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.190030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.213467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.214090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.214371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.214642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.215386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:31.215693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.217178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd889c3-3902-433e-b06f-8676f0082b75 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.223912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 87/181] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.229280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.229280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.229280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.245137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:31.246426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:31.266797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190670c-cf92-4d8e-90e9-f6d53b49bbe2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.268569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.305616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.306261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.306546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.306815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.307568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:31.307874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.309440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.316464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 95/182] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=07556e60b8b243f0844a294e71af2a62 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.317459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.339425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 88/183] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:31.340935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.428792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15c2d4-8b32-4a12-a2ba-fc18e7a81b12 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.457661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbefb1-9baf-42bd-a84f-958030a577ca None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.466037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.467142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.467142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.467305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] PATH_INFO: `/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.468456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c1bfd1c0e7420b9a8651065c3d719d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.469842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.470042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.476424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 96/184] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/users/b2c1bfd1c0e7420b9a8651065c3d719d => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.476424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.506215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.507216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.507681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.508149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.509475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:31.509995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.511537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.512596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.528147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 89/185] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.529430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.595078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7799ba-4432-4e02-8306-3b8156b0e536 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.630638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.631289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.631560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.631832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.642618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:31.647038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1719c-d019-478d-a3cc-d02576b685b6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:31.648132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:31.649011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:31.649864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 97/186] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:31.650503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.693819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.694809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.695223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.695723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.739675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.753903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.761686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86277a-20a6-456f-bfa7-787db153befd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.781247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.783785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.808615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.809270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.809873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.810152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.824453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.825997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.827704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 98/187] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.828471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.888939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 90/188] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:31.889790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.931416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac76cb-0af6-4f53-95b3-c7ba420c615d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:31.961093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:31.965216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:31.965829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.966107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.966377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.967042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:31.967341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:31.968776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f32295-3dd3-490a-aeb1-ac6e37aba08d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:31.976905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 99/189] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:31.977886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c2da-c8e8-4602-a8a2-46c1c2c6252f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:31.978162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c2da-c8e8-4602-a8a2-46c1c2c6252f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:31.978428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c2da-c8e8-4602-a8a2-46c1c2c6252f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:31.979621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 100/190] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:31.980308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c2da-c8e8-4602-a8a2-46c1c2c6252f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:31.995784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.000353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224fd15-e5f5-4e81-a2c6-35a07ef00d69 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.034654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.035221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.035507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.035788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.037052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.038511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.040114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 91/191] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:32.040901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.085587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c2da-c8e8-4602-a8a2-46c1c2c6252f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.121586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.122198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.122476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.122738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.141944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa82691-91d8-4962-a9fb-2c1e7454fd12 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.175659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.176292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.176635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.176909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] PATH_INFO: `/v3/projects/73d4df0979174d19936f9bf141b3b93d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.178052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] RBAC: Authorizing `identity:get_project(project_id=73d4df0979174d19936f9bf141b3b93d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.179512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.181208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 92/192] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/projects/73d4df0979174d19936f9bf141b3b93d => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:32.181963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.190423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.194312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.195800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d47a0-17d6-439c-a7ae-0dff39b4e8f6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.209976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.279530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d692b15-8123-4d37-aefe-d93b0042ea9f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.297797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 101/193] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:32.298956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.299238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.299506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.316235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:32.316691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.317353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.317455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:32.317635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.317909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.319056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.320527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.322139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 93/194] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:32.322870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.386195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 102/195] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:32.387029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.410983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.432187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.439987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaccae-f44b-4c60-be10-7bdc7f28a437 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.474358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.474975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.475248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.475792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.476156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.477642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.487750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a61797-50a2-4d87-b11f-d75aebba596f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.500871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 94/196] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:32 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:32.502113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.527506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.528153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.528434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.528740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.529501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=73d4df0979174d19936f9bf141b3b93d, user.id=b2c1bfd1c0e7420b9a8651065c3d719d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:32.529807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.531318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.538596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 103/197] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/role_assignments?user.id=b2c1bfd1c0e7420b9a8651065c3d719d&scope.project.id=73d4df0979174d19936f9bf141b3b93d&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:32.539891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.667309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b97218-f9b5-4618-8d29-9f83898a47a2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.682550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.692246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f84ce-5d95-4991-a941-3ee3a17fb955 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.720885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.721529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.721876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.722155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.730451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:32.737996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386045a-1dff-4699-a902-2240b278747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.738674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386045a-1dff-4699-a902-2240b278747d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.739312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386045a-1dff-4699-a902-2240b278747d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.740171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386045a-1dff-4699-a902-2240b278747d None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.741567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386045a-1dff-4699-a902-2240b278747d None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.743122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386045a-1dff-4699-a902-2240b278747d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.746299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 95/198] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:32.746604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386045a-1dff-4699-a902-2240b278747d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.813850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:32.815360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:32.873296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386045a-1dff-4699-a902-2240b278747d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:32.916150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:32.916830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:32.917109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:32.917377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:32.929585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:32.935922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:32.936845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:32.937630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 96/199] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:32.938274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.948497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 104/200] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:32.949242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:32.960101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.017338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.067161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134f38d-2b8b-42aa-ac5f-d0b219275c56 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.082762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f497-ae5b-4a5a-b1c0-cfe620993303 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.112866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.114489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.114920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.115330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.116199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=9678ab91407a45c7ba41ea834e760bbb) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:33.116690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.118275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.118707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.119460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.119807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.120143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.120933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:33.121294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.122756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.126458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 97/201] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/role_assignments?user.id=9678ab91407a45c7ba41ea834e760bbb&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:33.127439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:33.143993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 105/202] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:33.150170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:33.261590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.300765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e8299-5877-40a7-9fbd-694267dcbd16 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.334794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.335434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.335708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.335976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] PATH_INFO: `/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.337139 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] RBAC: Authorizing `identity:get_user(user_id=fa7bbedcf83448bc9417ba8cd9fcca8b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.338591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.344883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 106/203] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:33 2022] GET /identity/v3/users/fa7bbedcf83448bc9417ba8cd9fcca8b => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:33.346315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:33.352001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf87e41-3aca-4fe0-9642-4eda78cd9395 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.399871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.400440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.400752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.401031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.402174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.403654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.405618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 98/204] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:33 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:33.406103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.413142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:33.506390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b61c0-58a3-46db-be81-6ea9fc4500c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.513849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a08d-4574-4a5c-a4b0-a012f7c5e217 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:33.549190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.549819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.550106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.550384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.551130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=9678ab91407a45c7ba41ea834e760bbb) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:33.551472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.555444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.566055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 99/205] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:33 2022] GET /identity/v3/role_assignments?user.id=9678ab91407a45c7ba41ea834e760bbb&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=07556e60b8b243f0844a294e71af2a62 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:33.577643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:33.579080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:33.579363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:33.579637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:33.595635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.720661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:33.722582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:33.854803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 107/206] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:33.865650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:33.946732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:33.992871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babc99-e540-49a4-83a1-7f03d86a8b01 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:34.027670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:34.028314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:34.028620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:34.028898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:34.030040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:34.031501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a0d5b-5c43-4578-844a-6856bf52e322 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:34.033365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 100/207] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:33 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:34.039247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:34.080011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:34.145858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394f9e-74b4-40e7-a72a-4ad48a595f61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:34.198097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:34.198712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:34.198988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:34.199254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:34.199985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15, user.id=fa7bbedcf83448bc9417ba8cd9fcca8b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:34.200283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:34.201767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3776ce2-bf2f-4832-a6e0-9051695e7bf6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:34.213269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 108/208] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:34 2022] GET /identity/v3/role_assignments?user.id=fa7bbedcf83448bc9417ba8cd9fcca8b&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:34.890149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694949-cc77-47a2-acb9-ceff9b4b4f6a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:34.890685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694949-cc77-47a2-acb9-ceff9b4b4f6a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:34.890685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694949-cc77-47a2-acb9-ceff9b4b4f6a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:34.892298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 101/209] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:34.897962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf495-8176-46e9-98fd-445ff3bc976d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:34.898459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf495-8176-46e9-98fd-445ff3bc976d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:34.898918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf495-8176-46e9-98fd-445ff3bc976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:34.917479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3b8cf495-8176-46e9-98fd-445ff3bc976d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:34.919370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b8cf495-8176-46e9-98fd-445ff3bc976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:34.991270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 109/210] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:35.090677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850a6b1-0776-4a98-a974-7128a992d1b6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.091305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850a6b1-0776-4a98-a974-7128a992d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.091790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850a6b1-0776-4a98-a974-7128a992d1b6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.093434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 102/211] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:35.098967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c7ee3-2d68-4cab-be77-1f63fdaed068 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.099245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c7ee3-2d68-4cab-be77-1f63fdaed068 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.099734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c7ee3-2d68-4cab-be77-1f63fdaed068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.118407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-423c7ee3-2d68-4cab-be77-1f63fdaed068 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:35.119957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-423c7ee3-2d68-4cab-be77-1f63fdaed068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:35.147199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.190798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 110/212] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:35.262420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850a6b1-0776-4a98-a974-7128a992d1b6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.364651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850a6b1-0776-4a98-a974-7128a992d1b6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.370635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bfa6-1014-4a09-ad1c-2211562c334a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.370914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bfa6-1014-4a09-ad1c-2211562c334a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.371180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8bfa6-1014-4a09-ad1c-2211562c334a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.372686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 111/213] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:35.378577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.379346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.380173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.403706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:35.405228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:35.408193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.411136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.411779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.412058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.412327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.423861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:35.429854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:35.430740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:35.431496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:35.437234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 103/214] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:35.438272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.489979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 112/215] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:35.504747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.563256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeda162-03d3-4d99-a5cc-57cd9a21fa16 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.601626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.610566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91e60-2d79-4458-a5c4-594d2c005d97 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.615134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.615979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.616490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.616961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.617811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:35.618287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:35.620322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:35.632020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 104/216] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:35.639587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.649550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.650448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.650945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.651499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.652801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:35.654792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:35.656605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 113/217] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:35.662418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.689312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.744987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54f59e-af07-43d0-9196-65fce9a55a91 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.775185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d553e2b-390e-499e-8621-66bbd9b86c61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.780139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.780822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.781082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.781382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] PATH_INFO: `/v3/users/319918da0afe4e2094148b42caf9d5a2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.795695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] RBAC: Authorizing `identity:get_user(user_id=319918da0afe4e2094148b42caf9d5a2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:35.797201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794f244-f715-4ce3-9e50-7bd4158c3e1f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:35.798863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 105/218] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/users/319918da0afe4e2094148b42caf9d5a2 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:35.800007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727988a-6e25-4d60-b7ba-a3b30590eda5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.800284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727988a-6e25-4d60-b7ba-a3b30590eda5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.800560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727988a-6e25-4d60-b7ba-a3b30590eda5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.804687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 106/219] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:35.804687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727988a-6e25-4d60-b7ba-a3b30590eda5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.814322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.814949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.815241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.815511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.816167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:35.817714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:35.838537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 114/220] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:35 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:02:35.839367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:35.858769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.925455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727988a-6e25-4d60-b7ba-a3b30590eda5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.927334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:35.952232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda714b-621e-4429-abfa-e9004415efb5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:35.966365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:35.967026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.967308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.967580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.980148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f59e68fe-2d87-4508-af65-e26d998994ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:35.985101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:35.986068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:35.986939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 107/221] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:35.987734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:35.987734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:35.987880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:35.999222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.000087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.000534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.000984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.007130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:36.008325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:36.083239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.084885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.093067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 108/222] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:36.094069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.109361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.212007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 115/223] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:36.213847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.218199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.224226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad4922-46fb-41d3-90b7-d8a7bc45c286 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.268925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.269827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.270175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.270461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.271152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:36.271456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.273437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.284471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 109/224] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:36.291488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.331096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e94b9-b48b-4066-a5a1-5f19fbffc966 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.349169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.381248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.382349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.382349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.382349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.383841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.385454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.387261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 116/225] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:36.403520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.413687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674236b-71cc-4c19-a6af-4da2e4149eba None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.450178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.450871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.451148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.451412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.452616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.454188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.460650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 110/226] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:36.461682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.489161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.539882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3416efb-ce95-4171-920e-aab14bca1118 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.568786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951c953-ecbd-48d1-8b25-266c17a2cf5a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.592432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.593903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.594170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.594443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.610787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:36.611864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.612487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.612793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.613063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.613820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=319918da0afe4e2094148b42caf9d5a2) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:36.614131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.615584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:36.618068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:36.618916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:36.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:36.624653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 117/227] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:36.624653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.628867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 111/228] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/role_assignments?user.id=319918da0afe4e2094148b42caf9d5a2&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:36.629996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.694613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.754309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75b21-32f7-4f42-8a89-e66ac1e1efc5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.791766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.792537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.792843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.793113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.794689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.796161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.797922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 118/229] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:36.803594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:36.877582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.907667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3fb42-67e9-4cd5-8dac-a1ca6505828e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.908232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a416-2fab-4790-a2a6-3da90f358d59 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:36.940951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:36.948464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.949165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.949497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.949751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/9678ab91407a45c7ba41ea834e760bbb/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.952041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, user_id=9678ab91407a45c7ba41ea834e760bbb, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.954034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.970825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:36.971548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:36.971838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:36.972113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:36.972967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:36.973665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:36.978809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:36.983826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 119/230] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:36 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/9678ab91407a45c7ba41ea834e760bbb/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:02:36.998461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 112/231] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:37.005171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:37.142892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e1c37-4236-4f80-a2f1-48b4faca1957 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:37.190417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:37.191062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:37.191343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:37.191612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:37.192857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:37.194381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:37.196181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 120/232] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:37 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:37.205660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:37.321323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368be77-9fe0-4933-969b-83ceadab494a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:37.360022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:37.360698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:37.360979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:37.361248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:37.362391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:37.363889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100006ef-1880-4158-90f1-2b01f92ca1b5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:37.365591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 113/233] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:37 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:37.371585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:37.428918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:37.508693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eccae3-ff5c-48ad-b2ab-d40d7e89ef9f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:37.555046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:37.555809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:37.556089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:37.556362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/9678ab91407a45c7ba41ea834e760bbb/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:37.558640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, user_id=9678ab91407a45c7ba41ea834e760bbb, role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:37.560115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d157bc-e901-4c3c-b224-9e9cf7e66a24 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:37.602582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 121/234] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:37 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/9678ab91407a45c7ba41ea834e760bbb/roles/07556e60b8b243f0844a294e71af2a62 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:38.207874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984710-adb2-4478-a1c2-8390ed70c102 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.208400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984710-adb2-4478-a1c2-8390ed70c102 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.208400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984710-adb2-4478-a1c2-8390ed70c102 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.209954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 114/235] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:38.215450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454466-9dae-46c7-8ce0-bcd48bcd5aa5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.215954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454466-9dae-46c7-8ce0-bcd48bcd5aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.216427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454466-9dae-46c7-8ce0-bcd48bcd5aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.237476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-99454466-9dae-46c7-8ce0-bcd48bcd5aa5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:38.238751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-99454466-9dae-46c7-8ce0-bcd48bcd5aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:38.322780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 122/236] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:38.576692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b38663-0935-4a79-b6f5-cc1b0ecfecdb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.577357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b38663-0935-4a79-b6f5-cc1b0ecfecdb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.577830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b38663-0935-4a79-b6f5-cc1b0ecfecdb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.579643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 115/237] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:38.586282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4678c-2850-4cfa-acd9-9c0002706a9a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.587028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4678c-2850-4cfa-acd9-9c0002706a9a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4678c-2850-4cfa-acd9-9c0002706a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.605769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f6d4678c-2850-4cfa-acd9-9c0002706a9a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:38.607208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6d4678c-2850-4cfa-acd9-9c0002706a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:38.630752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:38.681375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 123/238] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:38.712974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b38663-0935-4a79-b6f5-cc1b0ecfecdb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:38.735042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:38.836822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b38663-0935-4a79-b6f5-cc1b0ecfecdb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:38.852116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888cf5-80a7-4072-ac43-774b7233b867 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.852405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888cf5-80a7-4072-ac43-774b7233b867 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.852713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888cf5-80a7-4072-ac43-774b7233b867 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.854344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 124/239] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:38.859224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.859509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.859780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.874600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:38.875242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:38.875522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:38.875798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:38.886631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:38.891654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:38.892538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:38.893325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 116/240] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:38.897874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:38.899080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:38.902922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:38.993570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.019854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f703d-83d5-4060-9325-7d0a1920314a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.043546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 125/241] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:39.057978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.058897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.059383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.059858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.060791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:39.061308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.062924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.073336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 117/242] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:39.079150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.106710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.249771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.266621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcb69-9ed0-4d46-8dfc-a3525c2b3008 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.269070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c1381-e9d3-47ff-bf55-5c12231aa5f1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.321487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.322547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.323025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.323489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] PATH_INFO: `/v3/users/717e8446e47f4dabb166047dc3ed8e7d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.337790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.338406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] RBAC: Authorizing `identity:get_user(user_id=717e8446e47f4dabb166047dc3ed8e7d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.338534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.338714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.338992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.340742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.347209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.347365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 126/243] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/users/717e8446e47f4dabb166047dc3ed8e7d => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:39.347444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.348801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 118/244] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:39.349403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.349576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.363873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.460623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeaf34-f5fd-48af-8737-200c467ef121 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.460956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d70e-71b1-4d38-a861-0f853789c03a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.496781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.497423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.497687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.497966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.499645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.500608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.501072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.501590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.512728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:39.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:39.519199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:39.520091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:39.522827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 127/245] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:39.524134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.537005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.570420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.572236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.581420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.640194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a81d81-9487-4e9e-af72-9378dcd307ef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.678017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.678624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.683362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.683362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.683571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 119/246] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:39.683649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.683717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.683782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a5e93-0473-4d5c-8704-6c48d862bf84 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.707925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 128/247] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:39 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:02:39.709659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1200e-889b-431e-9339-e3eb1cec5ab3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.710168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1200e-889b-431e-9339-e3eb1cec5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.710691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1200e-889b-431e-9339-e3eb1cec5ab3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.712177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 129/248] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:39.717558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1200e-889b-431e-9339-e3eb1cec5ab3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.793051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.825871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce92729-8ac9-4a13-acca-4bc442639f42 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.838529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1200e-889b-431e-9339-e3eb1cec5ab3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:39.839310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:39.876538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.877185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.877461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.877720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.878840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.880342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb33b1-77f6-480d-bff5-fd7ad6316811 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.881490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:39.882327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 130/249] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:39.883521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.883795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.884065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:39.888552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:39.898708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 120/250] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:39.899832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:39.908489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:39.909719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:39.990117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 131/251] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:39.991082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.022413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b19b25-5cb4-4609-800f-1d5e15561b0b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.063807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:40.068608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.069231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.069510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.069781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.070520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=717e8446e47f4dabb166047dc3ed8e7d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:40.070827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:40.072282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa47257-cd37-498c-a453-96c1f922167c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:40.080797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 121/252] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/role_assignments?user.id=717e8446e47f4dabb166047dc3ed8e7d&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:40.082135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc814d-65fe-42f2-8b97-dacddb9e9595 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.082542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc814d-65fe-42f2-8b97-dacddb9e9595 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.082892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc814d-65fe-42f2-8b97-dacddb9e9595 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.085030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 122/253] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:40.092527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.093067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.093511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.114961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc628b2d-606e-4129-9fc5-c6002e1da9b1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.127754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:40.129241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:40.155459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.156200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.156597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.156968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] PATH_INFO: `/v3/users/319918da0afe4e2094148b42caf9d5a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.158365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] RBAC: Authorizing `identity:get_user(user_id=319918da0afe4e2094148b42caf9d5a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:40.159995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:40.162203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 132/254] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:39 2022] GET /identity/v3/users/319918da0afe4e2094148b42caf9d5a2 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:40.168188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.174638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:40.241573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 123/255] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:40.295471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376212-ebbb-4d2f-84b8-d37651e0d601 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.325438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:40.331668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.332391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.332717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.333093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.335004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:40.336479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:40.338161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 133/256] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:40.344105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.379467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.519212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb462-dc2c-45ba-a72a-445858baf2e4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.556532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355dd5-0daf-4ae1-89c6-389f15c02d13 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.611607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.612308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.612636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.612912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/319918da0afe4e2094148b42caf9d5a2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.616712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, user_id=319918da0afe4e2094148b42caf9d5a2, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:40.618178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:40.638015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.638692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.638998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.639266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.655476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 124/257] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:40 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/319918da0afe4e2094148b42caf9d5a2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:40.659864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:40.666109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:40.667044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:40.667845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:40.671036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 134/258] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:40.677308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.723800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:40.758068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.855526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b302b5-5945-4ed3-9f88-4a7900d996c5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.862917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:40.913844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbc4e4-e016-4238-ae78-63056c4aecaa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:40.920869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.921814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.922298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.922730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.923575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:40.924041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:40.926142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:40.942882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 125/259] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:40.948032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:40.955509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:40.956146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:40.956418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:40.956714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:40.967385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:40.972211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:40.973265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:40.974015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:40.978299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 135/260] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:40.979507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.084798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23534124-27ed-47e7-bc9d-295ac6d0c224 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.102933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.108077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.122144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.122759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.123033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.123299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.123964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:41.124320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.125762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.134643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c6dc-db3e-45c7-a0fa-d67d58282b40 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.138919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 136/261] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:41.140171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.221796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.222459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.222746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.223026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.229191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.230672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.232633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 126/262] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:40 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:41.238231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.243777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63615a9-0c1f-458a-be6a-ae45bdd31d2e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.280201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.280869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.281144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.281409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] PATH_INFO: `/v3/users/9678ab91407a45c7ba41ea834e760bbb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.285168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] RBAC: Authorizing `identity:get_user(user_id=9678ab91407a45c7ba41ea834e760bbb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.286758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.288569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 137/263] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/users/9678ab91407a45c7ba41ea834e760bbb => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:41.299186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.386240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.429273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077b82-8c97-4d31-b0a3-c7e26fb06c2b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.444719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e1c24-5897-4cc9-b400-5c0913ecccee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.486489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.487499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.487943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.488375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.502602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.503237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.503519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.503793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.541024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.563063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.564920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.616758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.636900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.638329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.678398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 138/264] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 28 10:02:41.680331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.782652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4a6d-f379-4e2b-ba65-909d8d8b7409 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:41.821577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 127/265] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:41.827478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.828261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:41.829179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:41.829672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:41.830131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:41.831653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:41.833414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:41.840080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 139/266] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:41.841234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:41.866696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.940958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:41.996583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b387b4-6c17-4a1b-9094-074227334f8e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:42.010354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40650-b3c2-4e3a-908d-a6a965b4c2fd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:42.056205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:42.056943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.057232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.057498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.058224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=9678ab91407a45c7ba41ea834e760bbb) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:42.058525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:42.059974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e5eb3-0928-47ce-8f51-d27bfe144cba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:42.061253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:42.062204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.062784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.063251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.064619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:42.066267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:42.068194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 128/267] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:42.074677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 140/268] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:41 2022] GET /identity/v3/role_assignments?user.id=9678ab91407a45c7ba41ea834e760bbb&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=07556e60b8b243f0844a294e71af2a62 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:42.086706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:42.231040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7765762-24e5-4205-8593-b1ba54fb5091 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:42.252847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:42.276597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:42.277236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.277519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.277795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.278551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=9678ab91407a45c7ba41ea834e760bbb) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:42.278861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:42.280351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f1086-650b-498d-83ea-bd8b985384c4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:42.287491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 129/269] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:42 2022] GET /identity/v3/role_assignments?user.id=9678ab91407a45c7ba41ea834e760bbb&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:42.700688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c74e-165f-489a-9157-950880f04fd8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.701200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c74e-165f-489a-9157-950880f04fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.701273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c74e-165f-489a-9157-950880f04fd8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.703033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 141/270] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:42.708979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7bf3f-2748-499f-b3b0-6908938e7dc2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.709398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7bf3f-2748-499f-b3b0-6908938e7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.709586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7bf3f-2748-499f-b3b0-6908938e7dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.733935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-08e7bf3f-2748-499f-b3b0-6908938e7dc2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:42.735118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08e7bf3f-2748-499f-b3b0-6908938e7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:42.741994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01ce4-8ff3-4fe7-9395-0eef410f532a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.742273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01ce4-8ff3-4fe7-9395-0eef410f532a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.742539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a01ce4-8ff3-4fe7-9395-0eef410f532a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.744164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 142/271] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:42.757331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f0da-f08c-4a0d-9624-04ea746eb54d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.757643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f0da-f08c-4a0d-9624-04ea746eb54d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.757890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f0da-f08c-4a0d-9624-04ea746eb54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.784949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6370f0da-f08c-4a0d-9624-04ea746eb54d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:42.786383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6370f0da-f08c-4a0d-9624-04ea746eb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:42.824895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 130/272] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:42.871423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 143/273] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:42.912167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550166c4-ea98-4e57-839c-1a44162f9666 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.912448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550166c4-ea98-4e57-839c-1a44162f9666 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.912765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550166c4-ea98-4e57-839c-1a44162f9666 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.914274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 131/274] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:42.920019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:42.920311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:42.920630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:42.941440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:42.942726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:42.945672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.020361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 144/275] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:43.098857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550166c4-ea98-4e57-839c-1a44162f9666 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.148437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.209246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550166c4-ea98-4e57-839c-1a44162f9666 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.224424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.246768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.247657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.248069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.248459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.249747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.251390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.257669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b6b2a-9af8-4fcf-9e52-bf67e99fb42c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.257890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 132/276] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:43.259172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.292716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.293570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.294305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.294664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.303442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.306803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:43.312376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:43.313434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:43.314216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:43.320680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 145/277] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:43.322036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.363367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e425-7c42-4841-987b-3de7cd7d7aa7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.396066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.396709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.396985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.397627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.398281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.400097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.420877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193e1b7-6347-4e2a-9059-5acd0b7757f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.421044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 133/278] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:02:43 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:02:43.425503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.459821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.460469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.460774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.461044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.472041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.473993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-78068bb3-bf75-4df1-8d26-03d263c08efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:43.479064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:43.480035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:43.480829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:43.485376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 146/279] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:43.486838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54f35-bed4-4cbd-a575-6407abcdfeb6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.487126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54f35-bed4-4cbd-a575-6407abcdfeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.487402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac54f35-bed4-4cbd-a575-6407abcdfeb6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.489609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 147/280] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:43.490233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac54f35-bed4-4cbd-a575-6407abcdfeb6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.547596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d705c-ef76-4e43-9666-8fa174ac4261 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.569328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.591733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.592635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.593093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.593519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.594208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:43.594509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.595895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47346e40-ecab-44bf-aafc-032db781cd56 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.604949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 134/281] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:43.606245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.606930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.606930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.622399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac54f35-bed4-4cbd-a575-6407abcdfeb6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.625762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:43.626936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:43.660960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.660960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.661202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.661286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.662211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:43.662313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.663641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.673094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 148/282] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:43.673758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.715744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 135/283] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:43.717357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.762831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.794623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727821be-1323-4d37-86d6-8d827085fda9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.820202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:43.835499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.836496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.837007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.837365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] PATH_INFO: `/v3/users/717e8446e47f4dabb166047dc3ed8e7d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.839573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] RBAC: Authorizing `identity:get_user(user_id=717e8446e47f4dabb166047dc3ed8e7d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.841160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.843149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 149/284] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/users/717e8446e47f4dabb166047dc3ed8e7d => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:43.853471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:43.858304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0951b-1c9b-4d44-9f72-0c7d0f4db482 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:43.897633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:43.898974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:43.899421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:43.899854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] PATH_INFO: `/v3/users/061c799421b84e52b866cbad6e5c3645` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:43.919739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] RBAC: Authorizing `identity:get_user(user_id=061c799421b84e52b866cbad6e5c3645)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:43.921551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:43.923820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 136/285] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/users/061c799421b84e52b866cbad6e5c3645 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:43.929639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.025095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:44.044794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c870276-bb03-428d-bce7-34dd2d0e2923 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.056498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b586-420c-468e-8a3d-bd39cc1b999e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.089694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.090673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.091013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.091345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.092145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, user.id=061c799421b84e52b866cbad6e5c3645, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:44.092531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.094082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:44.094473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.095179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.095525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.095871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.097129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.098712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:44.099882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 137/286] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/role_assignments?user.id=061c799421b84e52b866cbad6e5c3645&scope.system=all&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:44.100927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.109031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 150/287] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:43 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:44.114708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.226567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e138ab-b8c0-492a-aa10-68f17c7ce430 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.265954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.267104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.270505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.270977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.285791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:44.293433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:44.301533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:44.302610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:44.303579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:44.308151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 138/288] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:44.314564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.338114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:44.370927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895422a2-0cb6-4918-8920-8a31e455d4c1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.435652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.436327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.436630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.436903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/717e8446e47f4dabb166047dc3ed8e7d/roles/9b8364ce17a6400b8f88cfbeee0a13f0` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.445955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, user_id=717e8446e47f4dabb166047dc3ed8e7d, role_id=9b8364ce17a6400b8f88cfbeee0a13f0)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.447371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:44.472301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d229d-82ba-4bf4-85f4-9fdecdf253c3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.491677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 151/289] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:02:44 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/users/717e8446e47f4dabb166047dc3ed8e7d/roles/9b8364ce17a6400b8f88cfbeee0a13f0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:02:44.508933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.509560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.509838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.510117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.510799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:44.511102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.512546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:44.523036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 139/290] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:44.528658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.674508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfacb20-2995-4389-b210-e338df3b125d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.695217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:44.714770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.716063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.716480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.716915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] PATH_INFO: `/v3/users/319918da0afe4e2094148b42caf9d5a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.718222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] RBAC: Authorizing `identity:get_user(user_id=319918da0afe4e2094148b42caf9d5a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.719858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:44.722049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 152/291] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:44 2022] GET /identity/v3/users/319918da0afe4e2094148b42caf9d5a2 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:44.728142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:44.811855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:44.844781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343a24c-c916-4bc5-9972-e0edefa4c177 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:44.878746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:44.879525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:44.879809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:44.880080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:44.957267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:44.958787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:45.049926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:45.075294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 140/292] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:45.080845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:45.230422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36452f5-4429-4baa-96e8-7533ad827171 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:45.266656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:45.267297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.267580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.267853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:45.269007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:45.270529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8c848-7e68-40aa-8e87-4fd4406869fc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:45.272406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 153/293] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:45 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:45.278445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:45.421011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e8ad-17df-4e73-8aec-0d30e60c9a67 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:45.460489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:45.461161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.461441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.461709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:45.462469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=319918da0afe4e2094148b42caf9d5a2) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:45.462781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:45.464280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976051ff-f31e-4c80-8c77-1ff196a1f533 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:45.473893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 141/294] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:45 2022] GET /identity/v3/role_assignments?user.id=319918da0afe4e2094148b42caf9d5a2&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:45.840007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aab97-7020-44fe-9f61-1cadd2bf10b1 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.840714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aab97-7020-44fe-9f61-1cadd2bf10b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.841197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0aab97-7020-44fe-9f61-1cadd2bf10b1 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:45.843085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 154/295] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:45.848362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4599-dae9-481c-a5c8-ecd5566967a1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.848859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4599-dae9-481c-a5c8-ecd5566967a1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.849035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4599-dae9-481c-a5c8-ecd5566967a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:45.874683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e19c4599-dae9-481c-a5c8-ecd5566967a1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:45.875875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e19c4599-dae9-481c-a5c8-ecd5566967a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:45.913102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:45.964392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 142/296] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:45.980060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ccf5-8f80-4597-a786-11d9c71d1661 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.980688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ccf5-8f80-4597-a786-11d9c71d1661 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.981159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ccf5-8f80-4597-a786-11d9c71d1661 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:45.982783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 155/297] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:45.988856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d74836-a7e9-4dd0-b115-e58f87ee3bb2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:45.989149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d74836-a7e9-4dd0-b115-e58f87ee3bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:45.989404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d74836-a7e9-4dd0-b115-e58f87ee3bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.010689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-91d74836-a7e9-4dd0-b115-e58f87ee3bb2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:46.011979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91d74836-a7e9-4dd0-b115-e58f87ee3bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:46.090597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 143/298] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:46.234754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ccf5-8f80-4597-a786-11d9c71d1661 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.294737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:46.340289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ccf5-8f80-4597-a786-11d9c71d1661 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.374916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.375927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.376411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.376902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.381496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d85b8-3661-4023-a968-e2bf46e05def None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.381783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d85b8-3661-4023-a968-e2bf46e05def None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.382057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d85b8-3661-4023-a968-e2bf46e05def None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.385254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 144/299] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:46.386273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d85b8-3661-4023-a968-e2bf46e05def None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.392559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c3a0c5bb-8294-45cd-83e3-cec6fae3d174 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:46.398147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:46.399181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:46.400131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:46.406441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 156/300] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:46.406441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.406441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.406441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.421804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:46.422978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:46.491550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:46.494183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 157/301] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:46.497482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d85b8-3661-4023-a968-e2bf46e05def None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.509080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.545773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.546658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.547080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.547997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.549767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:46.551912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:46.554036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:46.555542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 145/302] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:46.560780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.622150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4ed8e-4a60-46a3-b535-458305905097 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.657959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.658799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.659230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.659646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.660533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:46.660972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:46.662551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:46.671782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 158/303] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:46.677056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.696177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c9806-bbbe-44f5-a808-11915e478f69 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.737635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.738259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.738570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.738918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.739712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:46.741679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:46.762625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 146/304] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Aug 28 10:02:46 2022] POST /identity/v3/groups => generated 229 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:02:46.776266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.807748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:46.812127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929cc01-613e-4f89-a1e9-f4343521b049 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.821757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:46.852953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.853608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.853989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.854265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] PATH_INFO: `/v3/users/836440e281ef4fc38ba7e0b1affac195` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.871173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] RBAC: Authorizing `identity:get_user(user_id=836440e281ef4fc38ba7e0b1affac195)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:46.872834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d267bd-c5c2-4d4a-9603-b24d486a764c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:46.874735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 159/305] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/users/836440e281ef4fc38ba7e0b1affac195 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:46.875940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e2d8-03e6-4c16-95ac-e39050a924c0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.876054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e2d8-03e6-4c16-95ac-e39050a924c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.876297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e2d8-03e6-4c16-95ac-e39050a924c0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.879997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 160/306] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:46.880913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e2d8-03e6-4c16-95ac-e39050a924c0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:46.892302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaf514-37fe-4c50-a706-48ebefc01164 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:46.932394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:46.933309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.933736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.934150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.945125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e51cff84-eee6-4b57-ba5c-ee069231ca16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:46.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:46.951084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:46.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 147/307] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:46.952523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:46.953418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:46.953418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:46.971280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:46.973381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:47.006048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e2d8-03e6-4c16-95ac-e39050a924c0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.044711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 148/308] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:47.052729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.052857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, user.id=836440e281ef4fc38ba7e0b1affac195, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.052934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.058599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 161/309] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Aug 28 10:02:46 2022] GET /identity/v3/role_assignments?user.id=836440e281ef4fc38ba7e0b1affac195&scope.system=all&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:47.089482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:47.172715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb826-c0be-4165-94ab-ef75c606e214 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.215873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.216421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.216729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.217002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.217681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:47.217981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.219379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.228464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 149/310] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:47.234217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.271113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:47.349868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8dc96-559e-494f-89cd-3fefd2507f49 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.357802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.385724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.386361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.386636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.386906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] PATH_INFO: `/v3/users/061c799421b84e52b866cbad6e5c3645` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.388027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] RBAC: Authorizing `identity:get_user(user_id=061c799421b84e52b866cbad6e5c3645)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.389532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.391446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 162/311] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/users/061c799421b84e52b866cbad6e5c3645 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:47.396799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.450949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:47.481007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79654ee-99a3-4aec-bbd5-5e84b6b1cec5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.500692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:47.508518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a6fed-0761-4d52-a3f8-e91a4943183d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.527630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.529294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.529799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.530493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.544494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:47.551276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:47.552152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:47.553140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 150/312] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:47.558874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.559583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.559858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.560120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] PATH_INFO: `/v3/system/users/061c799421b84e52b866cbad6e5c3645/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.561873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.561979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=061c799421b84e52b866cbad6e5c3645, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.562975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.576582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 163/313] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Aug 28 10:02:47 2022] PUT /identity/v3/system/users/061c799421b84e52b866cbad6e5c3645/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:02:47.690268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7671df-c4e6-4322-bb02-50ea56289a63 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.728262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.728913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.729189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.729456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.730126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:47.730427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.731850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.740683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:47.741593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 151/314] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:47.746402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:47.848987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a554c-a702-4804-a554-a4eac517e445 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:47.883727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:47.884369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:47.884667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:47.884934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] PATH_INFO: `/v3/users/717e8446e47f4dabb166047dc3ed8e7d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:47.886054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] RBAC: Authorizing `identity:get_user(user_id=717e8446e47f4dabb166047dc3ed8e7d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:47.887521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:47.893464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 164/315] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/users/717e8446e47f4dabb166047dc3ed8e7d => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:47.895386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:48.001364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678476f-6116-4dd0-abca-c5e6cea723d5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:48.036615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:48.037499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.037918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.038331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:48.107412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:48.112513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:48.114110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:48.216789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 152/316] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:48.222532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:48.251466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:48.324345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94707ab5-dd34-4cf1-a54b-1e9ce01fde20 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:48.363435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:48.364131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.364407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.364704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:48.365819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:48.367291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6a99b-94d3-42eb-8934-a8997c4072f2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:48.369274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 165/317] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:48 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:48.374867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:48.473904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:48.479098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06817bde-3dfa-4358-ae20-7396b3b1a3d7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:48.515796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:48.516331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.516643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.516921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:48.517665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, scope.project.id=d466c1bf79b6424381e3ae8f4119989b, user.id=717e8446e47f4dabb166047dc3ed8e7d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:48.517973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:48.519442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cefeb-242c-46fc-a04f-02be01c491f3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:48.526033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 153/318] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:02:48 2022] GET /identity/v3/role_assignments?user.id=717e8446e47f4dabb166047dc3ed8e7d&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:48.768943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6886-703e-4fb2-89bf-68083fa1f931 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.769214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6886-703e-4fb2-89bf-68083fa1f931 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.769499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6886-703e-4fb2-89bf-68083fa1f931 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:48.770949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 166/319] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:48.776085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1cbc1-aa84-421b-99f4-bab93b74a09f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.776372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1cbc1-aa84-421b-99f4-bab93b74a09f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.776674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1cbc1-aa84-421b-99f4-bab93b74a09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:48.794868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e9e1cbc1-aa84-421b-99f4-bab93b74a09f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:48.796071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e9e1cbc1-aa84-421b-99f4-bab93b74a09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:48.863350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 154/320] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:48.999214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00718a-4539-4cd8-ab08-33cc6d96e514 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:48.999505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00718a-4539-4cd8-ab08-33cc6d96e514 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:48.999915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00718a-4539-4cd8-ab08-33cc6d96e514 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.001336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 167/321] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:49.006949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.007415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.007514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.025983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:49.027090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:49.094759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 155/322] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:49.128899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00718a-4539-4cd8-ab08-33cc6d96e514 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.229422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:49.238011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00718a-4539-4cd8-ab08-33cc6d96e514 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:49.271682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:49.272314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.272618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.272882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.273981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:49.275425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:49.277382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 168/323] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:49.282874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.308127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:49.373890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.385563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025bac8-51e9-44a9-be7e-7d96a60972b3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:49.422475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:49.423091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.423369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.423641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.424279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:49.425772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:49.452532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 156/324] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Aug 28 10:02:49 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:02:49.497638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e486b-f793-4c5c-92c1-b2ba4350cfcf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:49.536957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:49.537379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.537658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.537926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.549892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5fd47728-c1ab-4e69-9df7-08138cb168ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:49.555150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:49.556059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:49.557439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 169/325] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:49.564556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.597974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db416-8966-4315-9f96-32419354a00d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.598284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db416-8966-4315-9f96-32419354a00d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.598527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db416-8966-4315-9f96-32419354a00d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.600046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 170/326] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:49.605261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.605535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.605807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.625773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:49.626966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:49.649672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:49.660865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:49.685690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ece83-cf7f-418b-a781-04e963f38705 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:49.709583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 171/327] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:49.730373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:49.730997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.731274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.731602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.732286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:49.732663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:49.734057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:49.745634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 157/328] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:49.751096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.865686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b51b5e-2280-480c-99f0-8ffd7e1de96a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:49.904142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:49.904816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:49.905089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:49.905353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] PATH_INFO: `/v3/users/836440e281ef4fc38ba7e0b1affac195` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:49.906461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=836440e281ef4fc38ba7e0b1affac195)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:49.907911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:49.909698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 172/329] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:49 2022] GET /identity/v3/users/836440e281ef4fc38ba7e0b1affac195 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:49.915367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:49.986519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:50.018553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:50.027015 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e710ac-d5d4-4362-b5da-01a75b0ec737 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:50.063489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:50.064572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:50.065041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:50.065518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] PATH_INFO: `/v3/system/users/836440e281ef4fc38ba7e0b1affac195/roles/9b8364ce17a6400b8f88cfbeee0a13f0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:50.067201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=836440e281ef4fc38ba7e0b1affac195, role_id=9b8364ce17a6400b8f88cfbeee0a13f0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:50.068804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:50.082505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 158/330] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Aug 28 10:02:49 2022] PUT /identity/v3/system/users/836440e281ef4fc38ba7e0b1affac195/roles/9b8364ce17a6400b8f88cfbeee0a13f0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:50.116180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432112d4-ac3d-493b-ad6e-b8c73b5db0b0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:50.161960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:50.162729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:50.163016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:50.163498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:50.178117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:50.184273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:50.185206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:50.186000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 173/331] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:50.193062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:50.208087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:50.321199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ecd86-c5fa-462c-88e6-9ecc275c107c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:50.356170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:50.356809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:50.357087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:50.357358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:50.358025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:50.358320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:50.359692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:50.368293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 159/332] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:50.374015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:50.464938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:50.483976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34877-bacd-4c64-b290-2b40d6b9b7e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:50.519854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:50.520518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:50.520815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:50.521091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] PATH_INFO: `/v3/users/061c799421b84e52b866cbad6e5c3645` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:50.522265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] RBAC: Authorizing `identity:get_user(user_id=061c799421b84e52b866cbad6e5c3645)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:50.523752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eeaf85-0a16-46ea-a564-a04e0d773a91 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:50.525707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 174/333] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:50 2022] GET /identity/v3/users/061c799421b84e52b866cbad6e5c3645 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:50.531646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:50.639683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:50.674860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb154ac-b853-4519-ac83-a78def276c62 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:50.726156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:50.726781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:50.727056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:50.727324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:50.728062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] RBAC: Adding query filter params (role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, user.id=061c799421b84e52b866cbad6e5c3645, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:50.728364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:50.729856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f220ed-8d92-42ba-b3d3-255cb6bbe985 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:50.747360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 160/334] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Aug 28 10:02:50 2022] GET /identity/v3/role_assignments?user.id=061c799421b84e52b866cbad6e5c3645&scope.system=all&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:51.768347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b746f-fcba-46ff-8487-617d1dc8292d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:51.769420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b746f-fcba-46ff-8487-617d1dc8292d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:51.769933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b746f-fcba-46ff-8487-617d1dc8292d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:51.771814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 175/335] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:51.776879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a2b6-aa3d-4b9e-b639-0abb63fd90af None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:51.777284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a2b6-aa3d-4b9e-b639-0abb63fd90af None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:51.777452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a2b6-aa3d-4b9e-b639-0abb63fd90af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:51.805540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3355a2b6-aa3d-4b9e-b639-0abb63fd90af None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:51.806751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3355a2b6-aa3d-4b9e-b639-0abb63fd90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:51.891130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 161/336] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:52.335215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f084307-f1a7-482d-85c5-3d41d772935f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.335863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f084307-f1a7-482d-85c5-3d41d772935f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.336395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f084307-f1a7-482d-85c5-3d41d772935f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.338205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 176/337] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:52.344972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ee82-1135-4a88-ada9-1007df90132f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.345490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ee82-1135-4a88-ada9-1007df90132f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.345954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ee82-1135-4a88-ada9-1007df90132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.368684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bb99ee82-1135-4a88-ada9-1007df90132f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:52.370234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bb99ee82-1135-4a88-ada9-1007df90132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:52.430678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f084307-f1a7-482d-85c5-3d41d772935f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:52.451386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:52.469264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 162/338] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:52.556887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3dacf-76b3-4866-be37-6fbf9016fef3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.557208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3dacf-76b3-4866-be37-6fbf9016fef3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.557437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3dacf-76b3-4866-be37-6fbf9016fef3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.558925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 163/339] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:52.565535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.565815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.566085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.566260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f084307-f1a7-482d-85c5-3d41d772935f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:52.601676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:52.602880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:52.612295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:52.615915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.616471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.616972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.641468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:52.653221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:52.654157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:52.654915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:52.659115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 177/340] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:52.665207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:52.694015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:52.750178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 164/341] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:52.832343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d30d-349f-44ee-a6e0-ce05c10c2544 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:52.872190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243424de-974f-4906-826b-51c53f761490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:52.873227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243424de-974f-4906-826b-51c53f761490 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:52.873705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243424de-974f-4906-826b-51c53f761490 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:52.874217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243424de-974f-4906-826b-51c53f761490 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:52.875149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243424de-974f-4906-826b-51c53f761490 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:52.875645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243424de-974f-4906-826b-51c53f761490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:52.877275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243424de-974f-4906-826b-51c53f761490 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:52.886545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 178/342] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:52.891845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:52.912118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243424de-974f-4906-826b-51c53f761490 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.035879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ab92-5f72-4897-9da6-c4c1d149048d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.053840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.057445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.074835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.075470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.075756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.076053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.094049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243424de-974f-4906-826b-51c53f761490 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.137594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.138580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.139042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.139539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.153724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5c106545-6385-4642-8bb3-bb90d66d2360 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:53.164527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:53.165409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:53.166167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:53.166740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:53.167673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:53.169779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 179/343] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:53.171180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83addc-e796-45db-989b-e014c0b08081 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.171644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83addc-e796-45db-989b-e014c0b08081 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.172096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83addc-e796-45db-989b-e014c0b08081 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.177017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 180/344] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:53.182756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.183327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.183783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.204021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:53.205353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:53.286959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 181/345] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:53.287910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.289986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 165/346] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:53.290924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.313989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.391948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.410254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0fd7c-74ff-485f-8634-d8b6a8e9062f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.434330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde062d4-458c-41fd-91c4-d611297de629 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.448024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.448681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.448966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.449290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.450076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:53.450418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:53.451916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:53.461382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 166/347] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:53.462229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.481071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.481730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.482008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.482276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.498669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:53.503874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:53.504910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:53.505702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:53.509229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 182/348] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:53.510156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.557562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.577884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041ad84-0731-4a96-aeb7-a5d24acab8a4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.615076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.615730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.616014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.616287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.617526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:53.618990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:53.620856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 167/349] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:53.621509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.688091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45470-cb03-496e-8ea9-59236702d61f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.714446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.724911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4488-34cf-4d2f-9175-51fe6c3cd599 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.729624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.730248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.730566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.730839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.762989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.763937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.764232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.764800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.765500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:53.765967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:53.767967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:53.779473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 168/350] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:53.780653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.806219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.817266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:53.818859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:53.918318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3ae03-eaa7-49a3-ad6b-365d8049c7ab None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:53.948307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 183/351] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:53.949134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:53.973935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:53.974588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:53.975139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:53.975244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:53.981459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:53.987856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:53.993590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:53.994455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 169/352] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:53.995677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.080168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.088898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e501-04fd-452f-8b11-18e01be2ad68 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.121329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b05e8-28fe-41aa-85ea-d30a15175ccc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.130024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.130781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.131130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.131470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.138910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.140484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.142372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 184/353] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.143140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.165044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.165661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.165933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.166201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.245973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.260589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.260589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.262789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f6c3c-53f2-4eac-8acb-a517555bd58f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.304619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.305268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.305543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.305810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.306971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.308417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3730ca-f753-4760-af2e-8d253c85bbca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.310269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 185/354] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.311345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99bef-6efb-46e0-a946-a98d29b6f505 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.311627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99bef-6efb-46e0-a946-a98d29b6f505 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.312064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d99bef-6efb-46e0-a946-a98d29b6f505 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.318444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 186/355] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:54.318444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d99bef-6efb-46e0-a946-a98d29b6f505 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.345411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.382639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 170/356] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:54.383655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.447775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d99bef-6efb-46e0-a946-a98d29b6f505 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.487947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.488467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.489164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.489164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.490335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:54.490335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.491398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.500263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 187/357] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.501052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.502650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf9be-c246-4c4c-ba14-b3d03c4674fe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.520809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.542936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.543554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.543829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.544447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.545207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:54.545508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.547014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4f10-c279-4268-9434-b7e8b786743c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.560973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 171/358] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.562463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa791-cd64-4f4c-aaa1-446d94c8bc80 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.562753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa791-cd64-4f4c-aaa1-446d94c8bc80 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.563025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa791-cd64-4f4c-aaa1-446d94c8bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.582027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-354aa791-cd64-4f4c-aaa1-446d94c8bc80 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:54.583296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-354aa791-cd64-4f4c-aaa1-446d94c8bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:54.624546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.640243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc975c67-3a2c-474b-98ab-3751142a0d77 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.677901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 172/359] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:54.680918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d3a4d-b40b-438c-8249-0382e78aee1e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.681276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d3a4d-b40b-438c-8249-0382e78aee1e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.681819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d3a4d-b40b-438c-8249-0382e78aee1e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.683271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 173/360] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:54.684004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d3a4d-b40b-438c-8249-0382e78aee1e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.686831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.687588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.687870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.688143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.697434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.699417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.703423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 188/361] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.703423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.815241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d3a4d-b40b-438c-8249-0382e78aee1e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.820972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.842148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fd048-1f6e-4360-bbb0-7c1c7b4a4809 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:54.854658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.855453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.855738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.856010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.857181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.858646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd404a-aedf-4067-a9ed-6a198e15c17b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.860488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 174/362] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.861476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.861759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.862032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.886682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:54.887884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:54.888198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:54.888927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:54.889204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:54.889514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] PATH_INFO: `/v3/users/836440e281ef4fc38ba7e0b1affac195` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:54.891930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] RBAC: Authorizing `identity:get_user(user_id=836440e281ef4fc38ba7e0b1affac195)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:54.893513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:54.895266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 189/363] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/users/836440e281ef4fc38ba7e0b1affac195 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:54.897982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:54.922233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:54.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 175/364] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:54.969788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.037428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9c928-246b-4353-97e4-8ca11f78eab8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.075337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.075867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.076145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.076423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.077213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:55.077522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.079038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.081719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987d109-c098-4e54-b7c5-19b86a083e8f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.085438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 190/365] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=07556e60b8b243f0844a294e71af2a62 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:55.086299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.116653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.122063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.122731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.123005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.123282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.125221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.126777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.128827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 176/366] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:54 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:55.129706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.168189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.213409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7245d1-e5e6-4ff6-9f72-7b20baf14466 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.253305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.253948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.254342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.254502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468aeb-7fe0-44cc-bb02-6162ce127e06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.254651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.255513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] RBAC: Adding query filter params (role.id=9b8364ce17a6400b8f88cfbeee0a13f0, user.id=836440e281ef4fc38ba7e0b1affac195, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:55.256784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.259363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.277803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 191/367] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/role_assignments?user.id=836440e281ef4fc38ba7e0b1affac195&scope.system=all&role.id=9b8364ce17a6400b8f88cfbeee0a13f0 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:55.278931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.302706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.303407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.303691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.303967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.318637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:55.324285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:55.326457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:55.328338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 177/368] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:55.332997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.448965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.453624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb681412-eeb4-41ef-9dbd-25847c22f833 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.494283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.494914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.495588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.495878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.496666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:55.496992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.498421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.508299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 178/369] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:55.509834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.522953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.561706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cfff8-25b1-4c9a-b905-c49778d3fb02 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.604852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.605470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.605749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.606018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.606750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=d466c1bf79b6424381e3ae8f4119989b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:55.607054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.609564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.615776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 192/370] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 260 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:55.616635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.640445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159af1a-2ebc-412e-b5fd-e614723a2dc1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.686643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.687644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.687928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.688465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.710105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:55.717328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:55.719174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:55.721018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 179/371] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:55.728318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:55.747266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ce3e4-6de9-4b70-a389-e51101f62cee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.755470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.787736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.788373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.788879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.789159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.801095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:55.878855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.881964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:55.928236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a625e-d5e5-4c76-ad3f-9090a7a8bbcb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:55.992226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:55.992883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:55.993162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:55.993420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:55.994090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:55.994384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:55.995763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.010895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 180/372] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:56.017556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.049255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 193/373] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:56.057534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.137063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:56.188885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:56.216697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8973-b3d5-4b22-8646-8a4b48ff5595 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.253604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ab5a-b96a-4c82-a816-03b591d1dd8b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.265275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.265901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.266173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.343363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.345271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.368666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.369692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.370165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.370625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.376434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.378183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.380672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 194/374] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:56.386798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.451139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 181/375] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:02:56.455317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:56.459236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.598619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f985da3-536e-48be-8d5b-083697098ebf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.630061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd14413-dc53-4a03-866e-a56c0124ddc5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.644930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:56.663566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.664214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.664489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.664780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] PATH_INFO: `/v3/projects/371955716dbe4a589ec9dad82a9828ca` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.667096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] RBAC: Authorizing `identity:get_project(project_id=371955716dbe4a589ec9dad82a9828ca)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.668596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.671993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 182/376] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/projects/371955716dbe4a589ec9dad82a9828ca => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:56.680623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.703258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.703826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.704102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.704367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.706280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.707785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.710807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 195/377] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:56.716729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.755811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:56.824081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540edd9a-1ebc-4339-9691-4f9b2ed5f19c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.839944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251dff-df01-41f8-bcdf-5753fbaa3788 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:56.859489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.860101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.860376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.860675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.861410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=d466c1bf79b6424381e3ae8f4119989b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:56.861709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.863234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.870288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 196/378] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=07556e60b8b243f0844a294e71af2a62 => generated 260 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:56.886667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:56.887305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:56.887586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:56.887852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] PATH_INFO: `/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:56.894371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] RBAC: Authorizing `identity:get_group(group_id=5e1bb20767614bb4b4c2f581f31a5acb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:56.895928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362ea66-cb6a-42e8-a7c8-a7f60693a246 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:56.897721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 183/379] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:56.903972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:56.957810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:57.011805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a7dbd-4c1a-473b-b969-ab73f6de2f39 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:57.049015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:57.049686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.050214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.050214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.051024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] RBAC: Adding query filter params (group.id=5e1bb20767614bb4b4c2f581f31a5acb, role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=371955716dbe4a589ec9dad82a9828ca) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:57.051332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:57.052840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc8796-cae6-4974-86e5-9408fd7527f2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:57.059644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 197/380] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:02:56 2022] GET /identity/v3/role_assignments?group.id=5e1bb20767614bb4b4c2f581f31a5acb&scope.project.id=371955716dbe4a589ec9dad82a9828ca&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:57.242894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c6719-9670-4434-9c4c-5eadcfdd9b25 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.242894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c6719-9670-4434-9c4c-5eadcfdd9b25 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.242894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c6719-9670-4434-9c4c-5eadcfdd9b25 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.242894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 184/381] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:57.248997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d709-1485-4902-ad47-7a7e040d76c8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.249676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d709-1485-4902-ad47-7a7e040d76c8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.250151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d709-1485-4902-ad47-7a7e040d76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.272268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1df2d709-1485-4902-ad47-7a7e040d76c8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:57.273507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1df2d709-1485-4902-ad47-7a7e040d76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:57.348630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 198/382] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:57.406688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b14a0-9d94-4c89-bb2c-b54e4f9ce98a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.407008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b14a0-9d94-4c89-bb2c-b54e4f9ce98a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.407230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b14a0-9d94-4c89-bb2c-b54e4f9ce98a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.408928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 185/383] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:57.414056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.414338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.414607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.433976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:57.435158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:57.461939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:57.504821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 199/384] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:57.641878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b14a0-9d94-4c89-bb2c-b54e4f9ce98a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:57.767162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b14a0-9d94-4c89-bb2c-b54e4f9ce98a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:57.777095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:57.791138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:57.806394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:57.807032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.807310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.807580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.820169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:57.825520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:57.826356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:57.827140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:02:57.831994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 186/385] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:57.833152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:57.901384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265bde-90ac-4475-aae4-25b7de92b338 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:57.936928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:57.937564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:57.937843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:57.938107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:57.947983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebf806-705c-40a6-b751-660099a88a6b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:57.949744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:57.954746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:57.956563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:57.958081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 200/386] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:02:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:02:57.961964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:57.992954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:58.001212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.002165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.002681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.003153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.004050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:58.004651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.006214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.020064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 187/387] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:58.025850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.068399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd03f67-cee9-47a5-a80d-a16cb917268f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.103042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:58.105499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.106228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.106642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.107005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.107775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:58.108098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.112575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.119723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 201/388] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:02:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:58.125176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.214710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:58.228468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c055-e06a-44e7-ab75-600b75d91748 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.262713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15562fc-0325-4e92-a31e-2f3bd3d3ad70 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.263223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.263831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.264105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.264428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.265576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.267744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.269546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 202/389] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:58 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:58.276600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.314494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.315148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.315428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.315700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.316915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.318458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f8882-962b-437b-99de-06b39006c59c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.320563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 188/390] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:58 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:58.321767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49d469-3719-49ab-900e-af0b7ca2c63b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.322047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49d469-3719-49ab-900e-af0b7ca2c63b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.322317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49d469-3719-49ab-900e-af0b7ca2c63b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.323869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 189/391] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:58.328711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49d469-3719-49ab-900e-af0b7ca2c63b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.385171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4997122-478d-4f5f-bb0a-7c09008ae0f5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.420919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.421555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.421829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.422095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.423289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.424823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56968b-be95-43a3-a09b-df242d516917 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.426833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 203/392] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:58 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:02:58.432935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.433219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.433492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.438596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:58.440949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49d469-3719-49ab-900e-af0b7ca2c63b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.455077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:58.456271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:58.472042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:58.479547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.480194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.480471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.480769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.482870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.484431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.486586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 190/393] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:58 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:58.487601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.530833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 204/394] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:02:58.531659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.646376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3d99d-c949-43e8-881a-78c1762f2343 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.661641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa728e5-23df-4384-b94d-58fbee62b65b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:58.705337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.706022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.706293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.706564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/groups/ba54c95ab813490bb4d64634b809ce87/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.709167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, group_id=ba54c95ab813490bb4d64634b809ce87, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.710622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.715393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:58.716111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:58.716423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:58.716735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/groups/ba54c95ab813490bb4d64634b809ce87/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:58.721274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] RBAC: Authorizing `identity:create_grant(project_id=42a98ca7d20c444aa0782c5ffb7b5e15, group_id=ba54c95ab813490bb4d64634b809ce87, role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:58.722838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:58.737050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 205/395] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Aug 28 10:02:58 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/groups/ba54c95ab813490bb4d64634b809ce87/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:58.744788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 191/396] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Aug 28 10:02:58 2022] PUT /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15/groups/ba54c95ab813490bb4d64634b809ce87/roles/07556e60b8b243f0844a294e71af2a62 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:02:58.923000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:58.936803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:59.038679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ea13a-03a0-4bda-9bbd-c058dc36b0ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:59.080218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:59.080986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.081207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.081476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.093686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:02:59.098758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:02:59.100837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:02:59.102754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 206/397] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:02:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:02:59.108543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:59.135287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:59.210691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c0ddb-2327-4ebd-9811-f2ca74dc3983 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:59.245394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:59.246011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.246348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.246618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.247296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:02:59.247600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:59.249009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:59.257880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 192/398] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:02:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:59.263533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:59.357432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:59.400280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328a3a4-ec2d-4d72-8291-1ead00e1c489 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:59.451204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:59.452227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.452754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.453251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.454577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:59.456226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb03f-0dad-470e-bd55-02f9cdde3e04 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:59.458280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 207/399] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:02:59 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:59.464142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:59.597015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555346-d335-43b2-9dcc-c68fd2e47283 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.597429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555346-d335-43b2-9dcc-c68fd2e47283 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.597853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555346-d335-43b2-9dcc-c68fd2e47283 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.599659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 208/400] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:02:59.605713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25ba37-fd7e-4bd4-92f2-1375559723c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.606834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25ba37-fd7e-4bd4-92f2-1375559723c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.607398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25ba37-fd7e-4bd4-92f2-1375559723c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.628404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:02:59.642436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ab25ba37-fd7e-4bd4-92f2-1375559723c7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:59.644622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab25ba37-fd7e-4bd4-92f2-1375559723c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:59.647369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182294-5761-46ce-856e-fb3dc851ecc2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:59.701570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:59.702470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.702890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.703846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.706012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:59.708132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:59.711872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 193/401] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:02:59 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:02:59.717516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:02:59.770470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 209/402] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:02:59.841430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbabfe-0533-41ee-acb1-0e309d1563c8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.841695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbabfe-0533-41ee-acb1-0e309d1563c8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.841968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbabfe-0533-41ee-acb1-0e309d1563c8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.843572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 210/403] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:02:59.851223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.851509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.851784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.857705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc813-f940-4617-846b-a81d86e8da06 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:02:59.881501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:02:59.882695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:02:59.892913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:02:59.893582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:02:59.893862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:02:59.894135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/groups/ba54c95ab813490bb4d64634b809ce87/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:02:59.898988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, group_id=ba54c95ab813490bb4d64634b809ce87, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:02:59.900460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:02:59.926366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 194/404] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Aug 28 10:02:59 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/groups/ba54c95ab813490bb4d64634b809ce87/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:00.000974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:00.008879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 211/405] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:00.155198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.177352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:00.287760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a3b68-ffc3-4616-9aa9-73323791d8ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:00.324264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:00.324935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:00.325216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:00.325493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:00.335726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:00.340766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:00.341659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:00.342529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 195/406] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:00.353629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.386002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.522335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c420a-b9f5-4b23-83bf-ffc54d932cca None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:00.525741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:00.568561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557503c5-97aa-428e-9b98-c9e10f5b9405 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:00.605362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:00.606004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:00.606280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:00.606552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:00.617158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:00.622043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:00.622917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:00.623771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 196/407] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:00.625234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:00.625882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:00.626161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:00.626426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:00.627120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:00.627422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:00.628888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:00.632364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.638631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 212/408] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:00.653556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.772904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:00.789710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc5aa5-8d38-491d-8e09-2be595d61664 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:00.826341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:00.827151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:00.827542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:00.827908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:00.828722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:00.829121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:00.830600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:00.836862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:00.840242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 197/409] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:00.846025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:00.879007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe665-66bc-45d5-8c7c-0ee13595032b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:00.956361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:00.957312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:00.957587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:00.957856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] PATH_INFO: `/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:00.959667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] RBAC: Authorizing `identity:get_group(group_id=5e1bb20767614bb4b4c2f581f31a5acb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:00.961351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:00.962936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 213/410] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb => generated 229 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:00.968573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.033882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48323-33fe-42fd-8db2-fc98919e8c13 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.077639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.078488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.078785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.079061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.080457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.082741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4fd00-7b20-469c-9e10-ffa2217011b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.085112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 198/411] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:01.086747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e3af5-1912-4104-b29c-add6f8ea8a74 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.087030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e3af5-1912-4104-b29c-add6f8ea8a74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.087299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e3af5-1912-4104-b29c-add6f8ea8a74 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.087681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0206d73-7714-48d3-93d3-d1e8305f2dfc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.088772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 199/412] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:01.091953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e3af5-1912-4104-b29c-add6f8ea8a74 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.126538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.136706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.137653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.137993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.138323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] PATH_INFO: `/v3/projects/371955716dbe4a589ec9dad82a9828ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.140385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] RBAC: Authorizing `identity:get_project(project_id=371955716dbe4a589ec9dad82a9828ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.141946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c64fb-b0f8-436d-9998-0dcf4640575d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.144576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 214/413] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:00 2022] GET /identity/v3/projects/371955716dbe4a589ec9dad82a9828ca => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:01.145965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.146422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.146863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.173498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:01.175111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:01.180568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.232873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e3af5-1912-4104-b29c-add6f8ea8a74 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.249912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 215/414] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:01.253348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.271090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.271779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.272172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.272358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.273712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.275056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f217ce-f4d2-4817-820c-7e890a8250f9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 200/415] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:01 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c01a9-31fe-4930-9eec-ac57c770f0bd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c01a9-31fe-4930-9eec-ac57c770f0bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c01a9-31fe-4930-9eec-ac57c770f0bd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 201/416] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:01.283299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c01a9-31fe-4930-9eec-ac57c770f0bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.395810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c01a9-31fe-4930-9eec-ac57c770f0bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.400858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75497d1-6f28-4443-a469-a9f4063b2211 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.431491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.432096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.432382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.432695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b/groups/ba54c95ab813490bb4d64634b809ce87/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.435437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] RBAC: Authorizing `identity:create_grant(project_id=d466c1bf79b6424381e3ae8f4119989b, group_id=ba54c95ab813490bb4d64634b809ce87, role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.435723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.437539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002500f1-bca7-40a0-901a-db5ece76e7da None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.439742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.448818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e602b9-6043-4850-b749-48b894841514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.450783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e602b9-6043-4850-b749-48b894841514 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.451097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e602b9-6043-4850-b749-48b894841514 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.451369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e602b9-6043-4850-b749-48b894841514 None admin] PATH_INFO: `/v3/projects/371955716dbe4a589ec9dad82a9828ca/groups/5e1bb20767614bb4b4c2f581f31a5acb/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.454658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e602b9-6043-4850-b749-48b894841514 None admin] RBAC: Authorizing `identity:create_grant(project_id=371955716dbe4a589ec9dad82a9828ca, group_id=5e1bb20767614bb4b4c2f581f31a5acb, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.456090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e602b9-6043-4850-b749-48b894841514 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.460376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 202/417] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Aug 28 10:03:01 2022] PUT /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b/groups/ba54c95ab813490bb4d64634b809ce87/roles/07556e60b8b243f0844a294e71af2a62 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:03:01.461894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.462420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.462883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.484216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:01.485857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:01.487366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 216/418] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Aug 28 10:03:01 2022] PUT /identity/v3/projects/371955716dbe4a589ec9dad82a9828ca/groups/5e1bb20767614bb4b4c2f581f31a5acb/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:01.521686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e602b9-6043-4850-b749-48b894841514 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.573980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 203/419] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:01.694867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e602b9-6043-4850-b749-48b894841514 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.736649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.737291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.737560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.737821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.748844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:01.753830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:01.754659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:01.755465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 217/420] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:01.761276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.881487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:01.900085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6689-daed-4a9f-b98f-4dd8cefed7c7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:01.943440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.943667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:01.974291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:01.974837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:01.975113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:01.975377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:01.976055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:01.976352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:01.977751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:01.994374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 204/421] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:02.000020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.032765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adaf77-2cd9-4c68-b5d5-0125b1a1077e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.103310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.104386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.104892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.105354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.131056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:02.137619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:02.138471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:03:02.139243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:02.152878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 218/422] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:02.168384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.199205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c147a-f1aa-436f-b3d6-a871d2b58d39 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.233126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.233654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.233930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.234200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.289764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:02.292160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff6dc3-26e1-4dff-a237-6bb582ca4c86 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.307432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:02.309118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e00b48-759a-47c7-8e88-7511465b385f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:02.309575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:02.331312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.332090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.332433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.332810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.333542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:02.333921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:02.335489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8fcc5-8e16-40e2-92d1-b0d9c43c5aad None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:02.344904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 219/423] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:02.346312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901e2e-08b4-4236-bc70-f87abf6512cc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.346656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901e2e-08b4-4236-bc70-f87abf6512cc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.347014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901e2e-08b4-4236-bc70-f87abf6512cc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.352755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 220/424] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:02.353898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901e2e-08b4-4236-bc70-f87abf6512cc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.433695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 205/425] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 28 10:03:02.434983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.435269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.435544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.457040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:02.458329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:02.471571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901e2e-08b4-4236-bc70-f87abf6512cc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.514817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.515590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.515875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.516146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.527404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 206/426] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:02.533499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.556878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:02.567855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:02.597575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:02.599300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:02.723527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 221/427] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:02.729451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.788104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e426d-c116-46f0-b665-466ca7bf73f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.829770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.830414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.830835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.831111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.832273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:02.833915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:02.836052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 207/428] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:02.841002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.845336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:02.900435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b42bbc-2c96-4251-a276-e9685fba6364 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:02.936691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:02.937430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:02.937707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:02.938019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] PATH_INFO: `/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:02.939743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] RBAC: Authorizing `identity:get_project(project_id=42a98ca7d20c444aa0782c5ffb7b5e15)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:02.941439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:02.948304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 222/429] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/projects/42a98ca7d20c444aa0782c5ffb7b5e15 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:02.948304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:02.976136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.004164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74ab3-51c4-4670-91ab-07311808a2f2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.039390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.039941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.040223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.040492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.041642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.043076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.045427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 208/430] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:03.046324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.081725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cdc8f-73f8-4d5e-be49-19f91cb88f67 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.128158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.129488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.130438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.130924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.134781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.147030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:03.151979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:03.153085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 28 10:03:03.153838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:03.156369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f47af-a3f5-4603-b1f9-39adad233832 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.158185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 223/431] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:03.159058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.199639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.200680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.201205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.202078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.203449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.205521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.213162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 209/432] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:03.213162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.233992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.280086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddae65-76a8-4ba5-b28e-7342444776d6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.317571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.318092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.318620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.318734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.319430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:03.319734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.321448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.328204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 224/433] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=07556e60b8b243f0844a294e71af2a62 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:03.329316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.337674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4b0f0-1259-4a4c-bcf8-5deedc696da8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.410085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.411141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.411634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.412103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.413014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:03.413510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.415113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.428554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.433972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 210/434] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:03.439769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.471651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b6e1-daa5-4815-857f-78427c64e193 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.509180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.509793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.510065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.510327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.511047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:03.511346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.512941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb76a0e-af0a-4fa2-998f-bb3fb46a59bb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.519805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 225/435] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=42a98ca7d20c444aa0782c5ffb7b5e15&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:03.542606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.563967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b33bb4-608b-4fe0-82e5-62b6cf15ade0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.608130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.609152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.609571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.610017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.690447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.692327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed4e0-0ee4-4bb1-939b-5336772703bb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.741370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b1d2-5dbc-4a07-a7af-aa553bde0bfd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.741691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b1d2-5dbc-4a07-a7af-aa553bde0bfd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.741967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b1d2-5dbc-4a07-a7af-aa553bde0bfd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.743510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 226/436] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:03.750884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aeb7f-c1af-4b66-90e1-1bc11531c0b8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.751164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aeb7f-c1af-4b66-90e1-1bc11531c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.751434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aeb7f-c1af-4b66-90e1-1bc11531c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.772940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-704aeb7f-c1af-4b66-90e1-1bc11531c0b8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:03.774301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-704aeb7f-c1af-4b66-90e1-1bc11531c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:03.787892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.808909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 211/437] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:03.810258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b466-97af-4b79-81b4-ceec06f2c87a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.810548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b466-97af-4b79-81b4-ceec06f2c87a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.810814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726b466-97af-4b79-81b4-ceec06f2c87a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.812189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 212/438] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:03.814732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726b466-97af-4b79-81b4-ceec06f2c87a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:03.856859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 227/439] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:03.858394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.858678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.858961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.878516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:03.879778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:03.944158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726b466-97af-4b79-81b4-ceec06f2c87a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:03.948808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:03.970498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 228/440] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:03.986911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:03.987648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:03.987991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:03.988405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:03.990257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:03.991148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:03.997476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 213/441] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:03.998730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.108421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c2a09-675e-4017-a033-08c015a38cf7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.127117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.150648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.151743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.152221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.152491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.154298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.155754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.157675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 229/442] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:03 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:04.163394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.188912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.238474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.253559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057f762-60c8-442f-8b24-7d46e5ce1202 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.286426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.287052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.287327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.287594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.288391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d453e5-acbf-44ad-8aab-b87e83d6194f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.297594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:04.302879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:04.303763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:04.304588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 214/443] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:04.305234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.325806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.326406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.326678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.326943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.327659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5, scope.project.id=d466c1bf79b6424381e3ae8f4119989b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:04.327957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.329414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.335490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 230/444] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:04.336275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.427533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04d979-f8c1-4f5f-9bc1-d34d54c29128 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.442500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.469588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8262019-a61b-47fc-a865-023f76f2620c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.471134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.471787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.472075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.472351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.484591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:04.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:04.491081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:04.491947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:04.492763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 215/445] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:04.498505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.508859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.509583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.510996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.510996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.510996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:04.511220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.512457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.522358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 231/446] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:04.527903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.630239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706fb34-b8bd-4a56-ada5-9c55142995d6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.654626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6ece-af5a-4737-b0d9-d80a1fa0d52b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.668595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.670061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.670344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.670603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.671282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:04.671582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.673096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.683644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 216/447] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:04.688225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.696008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.696872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.697341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.697759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.720442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.746738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.768618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.770175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.794534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c316b-3115-42bc-af7a-ef43f79fe407 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:04.828664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:04.829348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:04.829559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:04.829828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:04.875637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 232/448] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 28 10:03:04.881640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:04.896762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:04.898241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:04.930308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.960290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:04.982412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176bc04-8d90-4b63-9570-9df9579f3f74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.002000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 217/449] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 28 10:03:05.007964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:05.016533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.017186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.017467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.017899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] PATH_INFO: `/v3/projects/d466c1bf79b6424381e3ae8f4119989b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.018852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] RBAC: Authorizing `identity:get_project(project_id=d466c1bf79b6424381e3ae8f4119989b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.020334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.022025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 233/450] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:04 2022] GET /identity/v3/projects/d466c1bf79b6424381e3ae8f4119989b => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:05.029099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:05.109001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c314f-d10e-482d-9dc1-04450f88ed82 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.127339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7070a5-8111-430c-a0f2-ff416f9f230f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.138823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:05.144632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.145258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.145530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.145781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] PATH_INFO: `/v3/projects/371955716dbe4a589ec9dad82a9828ca` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.146890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] RBAC: Authorizing `identity:get_project(project_id=371955716dbe4a589ec9dad82a9828ca)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.148335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.150010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 218/451] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:03:05 2022] GET /identity/v3/projects/371955716dbe4a589ec9dad82a9828ca => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:05.155627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:05.163305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.163944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.164219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.164489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] PATH_INFO: `/v3/groups/ba54c95ab813490bb4d64634b809ce87` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.165694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] RBAC: Authorizing `identity:get_group(group_id=ba54c95ab813490bb4d64634b809ce87)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.167158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.168779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 234/452] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:05 2022] GET /identity/v3/groups/ba54c95ab813490bb4d64634b809ce87 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:05.174980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:05.190566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:05.261513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8778c6a-6730-42ac-ae33-42c7f7544d37 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.274924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00466660-c2fd-40d9-a5ee-cffbd54ba810 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.297296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.297918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.298199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.298481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] PATH_INFO: `/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.299591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] RBAC: Authorizing `identity:get_group(group_id=5e1bb20767614bb4b4c2f581f31a5acb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.301150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.302719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 219/453] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Aug 28 10:03:05 2022] GET /identity/v3/groups/5e1bb20767614bb4b4c2f581f31a5acb => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:05.308788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:05.311635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.312244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.312533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.312813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.313547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] RBAC: Adding query filter params (group.id=ba54c95ab813490bb4d64634b809ce87, role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=d466c1bf79b6424381e3ae8f4119989b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:05.313851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.315295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a0285-7a1c-41fb-b633-8491d16d7582 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.321479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 235/454] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:03:05 2022] GET /identity/v3/role_assignments?group.id=ba54c95ab813490bb4d64634b809ce87&scope.project.id=d466c1bf79b6424381e3ae8f4119989b&role.id=07556e60b8b243f0844a294e71af2a62 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:05.378607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:05.416586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2354f-21b0-44c6-a858-6537dd8423a7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:05.454031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:05.454650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:05.454929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:05.455211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:05.455950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] RBAC: Adding query filter params (group.id=5e1bb20767614bb4b4c2f581f31a5acb, role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=371955716dbe4a589ec9dad82a9828ca) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:05.456253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:05.457782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facc81d-0419-4d39-80d6-666a7363ab4b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:05.464214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 220/455] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Aug 28 10:03:05 2022] GET /identity/v3/role_assignments?group.id=5e1bb20767614bb4b4c2f581f31a5acb&scope.project.id=371955716dbe4a589ec9dad82a9828ca&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:07.897236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15bbc-adf2-4cc0-bf82-d3094449bd31 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:07.897791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15bbc-adf2-4cc0-bf82-d3094449bd31 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:07.897791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15bbc-adf2-4cc0-bf82-d3094449bd31 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:07.899375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 236/456] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:07.918386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5fce3-5abd-47ef-b5b6-87ae11b476cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:07.919237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5fce3-5abd-47ef-b5b6-87ae11b476cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:07.919346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5fce3-5abd-47ef-b5b6-87ae11b476cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:07.949532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-54e5fce3-5abd-47ef-b5b6-87ae11b476cc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:07.950744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-54e5fce3-5abd-47ef-b5b6-87ae11b476cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:07.957605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c9e-be88-4b2d-8472-8293b53d676e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:07.957875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c9e-be88-4b2d-8472-8293b53d676e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:07.958147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c9e-be88-4b2d-8472-8293b53d676e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:07.959755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 237/457] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:07.973861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:07.974184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:07.974420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.009638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:08.011716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:08.035013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 221/458] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:08.065574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0edfa1-9512-4ca9-ac74-9269dec24359 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.065966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0edfa1-9512-4ca9-ac74-9269dec24359 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.066215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0edfa1-9512-4ca9-ac74-9269dec24359 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.067790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 222/459] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:08.089421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.089734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.089972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.103782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 238/460] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:08.121503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:08.122695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:08.197609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:08.210883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 223/461] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:08.367856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.379234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:08.488653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a460e-3aac-4cac-9898-4d5043065b98 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:08.506837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.550358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:08.551770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.552233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.552700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.571266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:08.572792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:08.593180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:08.594089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:08.594935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:08.594935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.594935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:08.594935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 239/462] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:08.595753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.632037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f9c99-a208-479e-bf9a-8fde64217559 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:08.658635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:08.668577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:08.669547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.669820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.670086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.681260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:08.682713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-df7b0b5c-ae41-4a59-9a67-616fe82301c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:08.697286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:08.698363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:08.699180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:08.699180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.699180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:08.699180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 224/463] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:08.699837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce45b3-a7f7-4c81-a750-e41606bb3817 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.700107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce45b3-a7f7-4c81-a750-e41606bb3817 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.700370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce45b3-a7f7-4c81-a750-e41606bb3817 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.701871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 225/464] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:08.702404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce45b3-a7f7-4c81-a750-e41606bb3817 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.709644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fef55-f901-43db-98af-ba2c9fa9e6a3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:08.744551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:08.745094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.745366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.745737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.749572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:08.758551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:08.760045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:08.775149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:08.776202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:08.777265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:08.777265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:08.777265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:08.777265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 240/465] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:08.777542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.811678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce45b3-a7f7-4c81-a750-e41606bb3817 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:08.847248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:08.847905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.848301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.848582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.849352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:08.849648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:08.851008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad94ad-bdf9-4ccb-b682-6cb3564b09b5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:08.867542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 226/466] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:08.867542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.867542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.867542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.882757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:08.883924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:08.886328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b230-c761-44f4-8ef1-f4dc502b0ab7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:08.900273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:08.925014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:08.925650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:08.926257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:08.926257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:08.926831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:08.927133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:08.928530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:08.950731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 241/467] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:08.950731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.958264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 227/468] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:08.964610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:08.992303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.065575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deceee6-3889-4327-8197-acd11e4f51bc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.082518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb2b5b-1363-413d-84c6-409d535ca41a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.102763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.103596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.104054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.104422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.105193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.106843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.123680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.124376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.124702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.124998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.126044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:09.126480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.128220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.140972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 228/469] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:08 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:09.141910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 242/470] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:09.142575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.147848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.220914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.300161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e41a55-db60-4ae5-8bfd-8052f1a9af7c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.337327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.345456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.346462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.346737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.347002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.347867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.350190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.350862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757d632-682c-4af7-9a07-8a9a7c16e533 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.376116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 243/471] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:09 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:09.376985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.393802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.395046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.395654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.396422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.397495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.399367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14911f-8009-4c61-954d-bf61e19c80ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.432083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 229/472] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:09 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:09.495050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34876ff-9d1d-45c6-a648-bede071b0dc6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.496134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34876ff-9d1d-45c6-a648-bede071b0dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.496618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34876ff-9d1d-45c6-a648-bede071b0dc6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.499560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 230/473] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:09.506430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.507110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.507937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.516832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.545559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc76b-1909-4c94-bc2a-92c65fbee763 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.549664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:09.551096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:09.600064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.601636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.601636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.601636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.615807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.617454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:09.635712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:09.636896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:09.637840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:09.637840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:09.637840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:09.637840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 244/474] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:09.643452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.649026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 231/475] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:09.750620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54be34-85a7-4b09-bcf8-728095b09fce None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.786689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.787308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.787580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.787847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.788517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:09.788829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.790289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:09.800107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.808892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 245/476] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:09.810897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.896870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:09.914591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6bee4-e963-442b-96cb-986b06fb58fa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:09.942988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:09.961012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:09.961012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:09.961012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:09.961012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:09.961721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:09.961721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:10.001285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 232/477] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:09 2022] POST /identity/v3/users => generated 249 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:10.096711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c590ad-644c-44d0-af06-9b863c7b4067 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:10.135077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:10.135700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:10.135976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:10.136245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:10.136990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:10.138413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:10.155767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 28 10:03:10.155767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 28 10:03:10.155767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [parameters: {'id': '8790f02e1b4149d5a40c46edfe0d8d3b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 28 10:03:10.155767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Aug 28 10:03:10.173052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return ref.to_dict() Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:03:10.173986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 28 10:03:10.174864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 28 10:03:10.175795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:03:10.176832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:03:10.177630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'id': '8790f02e1b4149d5a40c46edfe0d8d3b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:10.178383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 28 10:03:10.179149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:10.186088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 246/478] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Aug 28 10:03:09 2022] POST /identity/v3/roles => generated 144 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:10.187369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:10.300226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891fd77-ade7-46ca-bdb3-bb21a1e9244e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:10.330998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:10.336292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:10.337125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:10.337762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:10.338198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:10.349466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-88cf1216-9ce7-4956-8a1c-d48e3663c65e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:10.354872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:10.355752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:10.356542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 233/479] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:10.361701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:10.430683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:10.475025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa247e2d-19d4-4f3d-ad44-a562a9f2f50e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:10.512638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:10.513302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:10.513573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:10.513884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:10.514553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:10.514857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:10.516250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaee42-0010-4ee2-8fc1-ac3fcccee2a6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:10.524845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 247/480] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:11.536843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228239c-c2fe-43ab-bea6-bd97f19aa6ab None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:11.537401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228239c-c2fe-43ab-bea6-bd97f19aa6ab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:11.537401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228239c-c2fe-43ab-bea6-bd97f19aa6ab None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:11.538861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 234/481] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:11.544470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173f29-c920-41b4-ab6f-9aff1539590e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:11.544920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173f29-c920-41b4-ab6f-9aff1539590e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:11.545104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173f29-c920-41b4-ab6f-9aff1539590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:11.571989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ba173f29-c920-41b4-ab6f-9aff1539590e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:11.573191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ba173f29-c920-41b4-ab6f-9aff1539590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:11.663068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 248/482] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:11.776823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847970-6184-4ac8-8d97-49a9277db663 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:11.777470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847970-6184-4ac8-8d97-49a9277db663 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:11.777941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847970-6184-4ac8-8d97-49a9277db663 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:11.779719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 235/483] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:11.785145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af4842-c3b0-44bf-9ed4-53a33df28291 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:11.785632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af4842-c3b0-44bf-9ed4-53a33df28291 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:11.786109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af4842-c3b0-44bf-9ed4-53a33df28291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:11.810187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-34af4842-c3b0-44bf-9ed4-53a33df28291 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:11.811658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-34af4842-c3b0-44bf-9ed4-53a33df28291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:11.878642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:11.901824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 249/484] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:11.938195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847970-6184-4ac8-8d97-49a9277db663 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.007927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b88622-b3ed-4418-b8f1-acd5e2fea88f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.008589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b88622-b3ed-4418-b8f1-acd5e2fea88f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.009083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b88622-b3ed-4418-b8f1-acd5e2fea88f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.010873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 250/485] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:12.015987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.016594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.017056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.046761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:12.048434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:12.095819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847970-6184-4ac8-8d97-49a9277db663 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.151227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.152391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.153045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.154487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.160415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 251/486] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:12.170853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:12.176814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:12.177726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.178512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:12.181044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 236/487] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:12.189607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.235701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.293883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.346538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1410b-ba10-4ca0-8ce0-eb9acf5747c0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.370782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.378348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cadf1b-9c68-4845-9598-ea81e9ed3ae5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.408425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.409068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.409345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.409627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.410299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:12.410582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.411976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.429074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 252/488] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:12.434567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.440768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.441463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.441738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.441998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.459145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:12.465956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:12.466808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:12.468253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 237/489] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:12.468935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.570609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9041-f3fb-4bf0-bbe6-06b6ac0db9ac None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.602320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46eb8c-8b23-4a2a-8286-d8f38f7d1f10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.610486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.611347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.611791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.612228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.616943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.625854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.627546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.638874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.639509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.639784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.640051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:12.645182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:12.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:12.647198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:12.647198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:12.647198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:12.648714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 253/490] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:12.649614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.655495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:12.662608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:12.663729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:12.664747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:12.666148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 238/491] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:12.671372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.697415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.773958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7814831-a3d9-429e-b3a7-e46e1333ae94 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.777349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806da9a-d9f7-452c-adda-85800d87ed85 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.810656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.811163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.811281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.811519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.811793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.812446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:12.812654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.812757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.813140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.813555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.814161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b07a2-c199-4453-a62f-bd727cdd5de6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.814495 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:12.814926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.816736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.825313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 254/492] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:12.825998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 239/493] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:12.826452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.827290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa47f-5a1a-43e7-82cd-c9bd4c804a3b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.827574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa47f-5a1a-43e7-82cd-c9bd4c804a3b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.827845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baa47f-5a1a-43e7-82cd-c9bd4c804a3b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.832544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 240/494] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:12.832544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baa47f-5a1a-43e7-82cd-c9bd4c804a3b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.861130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.932623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baa47f-5a1a-43e7-82cd-c9bd4c804a3b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.935540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:12.946839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d95e5-b2c8-4d99-9d37-38a8b4f73c8f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:12.973207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.973945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.974221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.974489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.981148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:12.981760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:12.982039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:12.982135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.982308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:12.982979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:12.983282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:12.983496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.984683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:12.997191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 255/495] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:12.997191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7800f866-447e-4d4e-bb56-4d4280f036fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:12.997457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:12.998471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:12.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 241/496] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:13.000005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.000005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.005022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.021398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:13.022770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:13.091902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 242/497] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:13.093349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.104812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c13638-6afe-4a20-bd28-be578b2839e1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.117283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.140050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.140857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.141142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.141410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.152884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.154355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:13.169834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:13.170735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:13.171560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:13.171560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:13.171560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:13.171560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 256/498] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:13.172016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.181196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.203369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80797477-3be3-4997-93da-c0aedd2938e2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.238445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.239087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.239368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.239645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.249934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.251397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3e14b25c-d45a-43e6-8c53-cd88f18f5128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:13.268742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:13.269707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:13.270532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:13.270532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:13.270532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:13.274887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 243/499] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:13.275992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628039e-ac7a-4132-9a0e-a622cdf5dfd2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.276280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628039e-ac7a-4132-9a0e-a622cdf5dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.276579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628039e-ac7a-4132-9a0e-a622cdf5dfd2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.277976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 244/500] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:13.278526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628039e-ac7a-4132-9a0e-a622cdf5dfd2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.281315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f302-0824-473c-8089-8ce3d1376b9a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.317490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.318179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.318528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.318871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.319673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:13.320068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.321626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.356944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 257/501] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:13.357859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.367074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.392389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628039e-ac7a-4132-9a0e-a622cdf5dfd2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.420466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.426644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.427330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.427676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.428018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.428796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:13.429117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.430494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f0ef9-99ef-46ce-9e48-41e127aa8db5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.441711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 245/502] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:13.442870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.443149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.443410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.458148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0375dca-a9ce-4179-b27e-bb03a55b373f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.461006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:13.462217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:13.492462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.493172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.493437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.493702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.494356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:13.494656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.496201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.507547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 258/503] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:13.508343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.528316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 246/504] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:13.529202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.604059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.619236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a3b-f80d-4e62-aecf-e463b30403ed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.643414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.646312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce12c41-5058-4cd3-97fe-1e4d5d5f947b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.655326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.656305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.656625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.656903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.668495 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:13.674171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:13.675074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:13.675928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 259/505] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:13.676631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.688666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.689491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.689618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.689791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.758505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.759976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.784122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12511d3-1ebd-4ff1-bece-97dac3770349 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.819963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.821313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.821944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.822244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.822521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.829923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.831372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:13.843708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:13.844702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 260/506] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:13.845582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.861190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 247/507] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:13.862034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:13.894864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:13.948825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b86fbc-e5b2-47e6-9a8c-c2873fb19c71 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.962791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10a792-4c6b-4170-8eb9-806d9597bfb3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:13.981169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.981788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.982068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.982346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.989789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.991246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:13.995741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:13.996351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:13.996667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:13.996939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:13.997606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:13.997909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:13.999268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:14.002306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:14.003308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:14.004266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 261/508] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:14.004840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.009708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 248/509] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:14.010577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.037969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.103543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa2f-5334-40f1-9caf-b240f7e4618d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.106110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.107584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da66192-a985-438e-9069-6e0ff2139748 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.139232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.139660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.139850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.140123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.140261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.140388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.140567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.140838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.141572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:14.141871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.143321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.151132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.152548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:14.166820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:14.167835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:14.168700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:14.168700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.168700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:14.168700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 262/510] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:14.169534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.180771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 249/511] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:14.181620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.266164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.270580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406af4c-7c61-47f9-ae92-cc041cefcf5e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.282207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ffdca-adf8-4f3a-b731-b08757b46fd4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.303707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.304326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.304625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.304888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.315129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.315653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.316279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.316578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.316672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.316855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.317590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:14.317889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.319310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:14.330311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:14.332279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:14.332279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.332279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:14.332279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 263/512] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:14.332608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.335872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 250/513] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:14.336884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.348110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.432642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c210ba4-b644-4e9a-9b33-ed5adfb81216 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.435466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298507-f8be-4f6e-a893-dbff379debad None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.466705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.467306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.467588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.467858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.468036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.468536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:14.468679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.468840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.468954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.469219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.470214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.479323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.480812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.481521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 251/514] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:14.482304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.488227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:14.495354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:14.496403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:14.497271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:14.497271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:14.497271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:14.497271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 264/515] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:14.497271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.557421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.590996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2943-4b4c-47db-b382-f1b5b10f8284 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.601711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5154e-c50d-418a-9b13-74400516b299 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.627134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.627749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.628026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.628297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.634290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.634890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.635163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.635424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.636060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:14.636354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.637764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.649518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 265/516] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:14.650291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.700238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.701657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.703496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.747326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a934f-3db2-457b-bc77-b12bea00f43c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.767297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.781427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.782036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.782315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.782587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.803835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 252/517] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:14.804932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.853576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.855061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.900904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960246c0-f16b-473e-9577-427f3d29fba0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:14.902398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:14.935937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:14.936540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:14.936838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:14.937145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:14.937745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:14.939188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:14.954648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 266/518] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:14.955553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.963741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 253/519] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:14 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:14.964582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:14.977697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.070550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0be33-c337-47b7-9adf-f0addcea7728 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.087940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1670e9-fe23-4ddb-9e5b-47ba4fd69286 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.109066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.109683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.109960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.110226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.110874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.111178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.113147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.124442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 267/520] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.125118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.127159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.127800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.128076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.128342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.135545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.137045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:15.149848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:15.150898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:15.151758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 254/521] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:15.152232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.175543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.217443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.236716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b200bb0-336e-484d-8e05-e4a88ec1520e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.263704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35789c43-7dc6-4f3b-8569-6ed2e8c89a8d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.270045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.270686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.270964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.271235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.282542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.284005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:15.299153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:15.300240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:15.301172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:15.301172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:15.301172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:15.301172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 268/522] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:15.301172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.303233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.304055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.304337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.305478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.317530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.319014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:15.333629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:15.334564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:15.335512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:15.335512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:15.335512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:15.335512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 255/523] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:15.335512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.406442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c38efe-3455-4617-a9b1-25ea19278c73 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.409514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.439383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.441406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2e9b2-80a8-4d27-bf6e-03d40a137e78 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.446384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.447184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.447600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.448024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.456790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.458491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:15.471170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:15.472225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:15.473389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:15.476830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 269/524] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:15.477648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.479690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.480470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.480778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.481858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.482577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.482885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.484335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.508470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 256/525] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.508470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.596938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810453e-97df-46d6-8771-9558ba6c384c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.610412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f06c-a711-4794-9b66-bc443cb0dfd9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.633753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.634606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.634606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.635195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.635506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.635803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.637208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.640755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.647254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.647974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.648254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.648542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.649193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.649491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.655239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 270/526] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.655380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.655458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.662814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 257/527] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.667302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.709633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.758986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29be8a-3224-4b6a-93a4-b07553eb540d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.769493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751302-2623-481e-91dc-fc70fd91770e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.795623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.796221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.796496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.796783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.797547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.797797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.799296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.803215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.803814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.804092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.804359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.805103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:15.805408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.806825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:15.824554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 258/528] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.824554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.830473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 271/529] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:15.831506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:15.889245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.938637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751e49-13da-4b56-bd03-2a5e7149feac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.971554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd051651-e4e8-431a-9164-497fe033c3c4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:15.985081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:15.986001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:15.986901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:15.987533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:15.987708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:15.995609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:15.998028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:16.010121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:16.011124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:16.011928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 259/530] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:16.012372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.019831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.020535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.020862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.021127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.034347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.035857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:16.048473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:16.049344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:16.050176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 272/531] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:16.051194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.139242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc49e0-cf90-4dab-be92-fa080862f027 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.168901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:16.177755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4206544-38ed-4ca6-bb2b-75cf0bf949ab None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.181495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.182110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.182409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.182682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.213734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.214373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.214791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.215060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.216083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:16.216699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.218211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.241046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 273/532] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:16.241932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.260295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.262145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.280190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:16.375267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 260/533] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:16.376715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.378566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9418f2-6be4-47a1-910f-aaf4370d31a7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.400067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:16.429811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.430705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.431130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.431545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.432418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:16.432911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.435021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.454554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 274/534] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:16.456050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.515126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d742c-8eb1-4921-ba1b-fbb6f4669255 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.553458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:16.565439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dcd36-c16a-455e-bfe8-2b3045230a44 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.573200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.574457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.574874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.575292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.576628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=32fb55fa700442dd974594ada94e7bb0) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:16.577075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.579090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.586299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 261/535] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/role_assignments?user.id=32fb55fa700442dd974594ada94e7bb0&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:16.587290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.637331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.638034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.638258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.638523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.665762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.667269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:16.706997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:16.707945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:16.708819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:16.708819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:16.708819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:16.708819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 275/536] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:16.713763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.758207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:16.783570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa928d12-816b-4df7-bfd5-b28aef70ec30 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.839932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.840567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.840848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.841115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.841954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=77f82ddcd0604a3697dc8b571719e1b6) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:16.842263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.843696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.849843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 262/537] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/role_assignments?user.id=77f82ddcd0604a3697dc8b571719e1b6&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:16.864734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2db46-988a-4882-af74-ec5d3e8abee8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:16.908082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:16.908742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.909018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.909291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.909976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:16.910274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:16.911663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408d925-a8ae-4bb6-91c8-ab96ba492218 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:16.929341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 276/538] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:16.934572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:16.982603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cee62-9e79-457c-8040-3910bb29e71a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.983241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cee62-9e79-457c-8040-3910bb29e71a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.983827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cee62-9e79-457c-8040-3910bb29e71a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:16.985901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 277/539] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:16.992069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:16.992712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:16.993180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:17.000615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:17.033753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:17.035219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:17.054297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69e8cb-dddc-4746-a89f-27f3c50aaa9f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:17.091073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:17.091950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:17.092388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:17.092830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:17.100590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:17.102383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:17.115804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:17.116741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:17.117560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:17.120002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 263/540] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:17.125410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:17.130753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:17.196068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 278/541] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:17.293073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1afb9-ad6a-4875-90f8-01a0ddb2e46d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:17.347034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:17.347675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:17.347950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:17.348218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:17.348939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:17.349240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:17.350675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:17.365916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 264/542] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:17.371944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:17.510907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1341f-462c-46b6-9812-900b9ee14054 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:17.540589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:17.550007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:17.550622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:17.550903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:17.551219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:17.552290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7f71b5efa95b41e196a505798527904d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:17.552945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:17.554770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:17.563717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 279/543] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:17 2022] GET /identity/v3/role_assignments?user.id=7f71b5efa95b41e196a505798527904d&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:17.575174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:17.677482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:17.731408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdfd66-ad78-4141-ab66-c3deba88c9ee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:17.770693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:17.771326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:17.771607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:17.771880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:17.793503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-00ac571a-27eb-4cde-99b6-05abd67ff528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:17.799104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:17.799941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:17.800882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 265/544] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:17.806232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:17.971349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf4c84-6cd1-48dd-80a4-305cbfb1a02d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.016017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.016732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.017337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.017337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.018010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:18.018302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:18.018871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898bd57-395a-4020-82cc-5e116aff80e1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.019153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898bd57-395a-4020-82cc-5e116aff80e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.019433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898bd57-395a-4020-82cc-5e116aff80e1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.019727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:18.020991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 266/545] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:18.027147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.027675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.028137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.036540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 280/546] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:18.038067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.058667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:18.059777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:18.158875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:18.188391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 267/547] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:18.218340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4c24-5c3d-476f-af57-a94407a924a5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.259945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.260972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.261443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.261902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.279810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:18.281599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:18.299241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:18.300166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:18.301171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:18.301171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:18.301171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:18.302957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 281/548] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:18.309057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.373543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:18.430665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d0ed5-db58-412b-87dc-f9d721bdc55d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.471530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.472147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.472480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.472786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.473443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:18.473746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:18.475129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:18.495207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 268/549] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:18.500966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.549607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.624413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2df-2cb9-4c53-b099-f537d4483fa3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.635483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:18.675194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.676058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.676454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.676843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.694310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:18.695771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:18.715131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:18.716686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:18.717770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:18.719282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 282/550] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:18.723301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb5b6-1bc1-47ab-8d88-c807afbbd2ac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.730796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.748863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:18.780557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.781220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.781476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.781717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.797843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:18.811430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:18.812382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:18.813404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 269/551] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:18.819083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:18.883427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496cf10-eede-432a-85a0-a35680fd4a90 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.928361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:18.929014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.929320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.929994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.930738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:18.931041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:18.933223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7170d7-9666-4e0c-8f16-be93922f7fe2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:18.957299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715e585-d252-42b7-9975-5dcfb31dbd2f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:18.960580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 283/552] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:18.962186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bb64-02fa-4b6a-8523-d7247e5dcbfe None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:18.962467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bb64-02fa-4b6a-8523-d7247e5dcbfe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:18.962738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bb64-02fa-4b6a-8523-d7247e5dcbfe None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:18.964286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 284/553] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:18.965700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1bb64-02fa-4b6a-8523-d7247e5dcbfe None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.008069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.008421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.009527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.009884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.010202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.011143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:19.011604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.014514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebfc6e-5660-4e77-abbb-dc576816b830 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.029776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 270/554] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:19.029776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.029776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.029776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.049371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:19.051005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:19.083933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.107190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1bb64-02fa-4b6a-8523-d7247e5dcbfe None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.138281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 271/555] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:19.139703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.141928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.142615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.142960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.143296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.211702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.213298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540ac0-e053-417b-9390-5ebc3f553c9a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.240682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.245343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29034de5-5219-429a-8cf6-b3273bf6a62f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.282380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.283412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.283901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.284354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.295865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.297625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2b30b9c0-20b4-44bf-9bbd-0bec60076d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:19.318408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:19.319375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:19.320264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:19.320264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:19.320264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:19.320427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 285/556] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:19.320427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446d4e-a721-41dc-bae2-15a8d5a45b52 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.320427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446d4e-a721-41dc-bae2-15a8d5a45b52 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.320427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00446d4e-a721-41dc-bae2-15a8d5a45b52 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.322202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 286/557] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:19.323286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 272/558] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:19.328643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446d4e-a721-41dc-bae2-15a8d5a45b52 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.329191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.329535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.329867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.348800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:19.350089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:19.362302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.423701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 273/559] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:19.425086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.444486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00446d4e-a721-41dc-bae2-15a8d5a45b52 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.487322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.499161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.499856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.500134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.500414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.513711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.515701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:19.536539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:19.537484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:19.538406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:19.538406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:19.538406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:19.540637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 287/560] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:19.541684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.545368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cf293-9bdc-41b3-b9fc-83e03284a564 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.588257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.588910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.589180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.589451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.590639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:19.590639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.591759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.605528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 274/561] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:19.606680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.616008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.685895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38750f9-826f-4225-bbd5-5422628f0ff5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.713923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23963d-ebd7-4500-9d6a-e625a7fd76b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.731589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.732118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.732395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.732695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.743129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:19.748471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:19.749378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:19.750187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:19.750855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.750855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.750855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.750855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.751686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:19.751686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.752447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.755426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 288/562] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:19.756073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.765739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 275/563] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:19.766528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.805227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.838682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:19.866145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9454a1-901b-420f-a9bd-cd5d56769fe5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.876652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1c0c6-5cbc-4415-acf7-e65d0e1d995c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:19.900981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.901613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.901893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.902165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.909730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:19.911183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:19.911553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:19.912171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:19.912450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:19.912739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:19.922715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:19.923733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:19.924710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:19.925249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:19.925319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 289/564] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:19.925319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:19.927970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:19.928879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:19.929710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 276/565] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:19.930383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.028144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.031168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1b035-c1e7-4254-b7df-3b7df932ef1d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.033396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56365af-ecd3-4909-9c8f-376eb3fdba6b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.060311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.067068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.067670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.067941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.068205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.068909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.069206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.069377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.070008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.070289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.070560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.070652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d9a1e-73f6-4061-9cd6-a75cb36c8ab0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.077105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.078395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 290/566] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.078546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.079520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7da6c-aa4d-41cc-a60f-e9cabc54dadd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.079797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7da6c-aa4d-41cc-a60f-e9cabc54dadd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.080069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7da6c-aa4d-41cc-a60f-e9cabc54dadd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.081431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 291/567] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:20.081941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7da6c-aa4d-41cc-a60f-e9cabc54dadd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:20.091335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:20.092398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 277/568] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:20.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.189683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7da6c-aa4d-41cc-a60f-e9cabc54dadd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.194863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99fc56-c311-4a03-8788-3785c0970f32 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.225174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.225784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.226063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.226331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.227053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.227162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.227356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.227650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.227927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.228201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.228876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d58aa-e860-4fa6-924d-8ca9641ba8d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.228987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.229177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.230521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.238220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 278/569] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.238991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.246220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 292/570] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.247404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.247687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.247958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.265410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:20.266569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:20.280797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.293450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.332776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 293/571] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:20.333589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.345190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ef3d0-6a98-4c68-88bd-5983313da3fd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.382411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.383145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.383422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.383688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.394402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.395949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:20.411025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:20.411962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:20.412867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:20.412867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.412867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:20.412867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 279/572] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:20.412867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.449845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f0a0-7f68-459e-ba7d-cf3af8526e22 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.484065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.484726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.484964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.485262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.485961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.486243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.487707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.504552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 294/573] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.504552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.514527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.528626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.532188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14dc86-77ba-4250-86bc-8de28b0467d1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.567914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.568684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.568962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.569232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.580048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.584063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:20.603104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:20.604144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:20.605024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:20.605024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.605024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:20.605024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 280/574] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:20.605713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.611169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20ff68-8819-4b65-9637-d8bba46e8bfb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.645066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.645694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.645967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.646232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.656562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.658037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:20.672829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:20.673914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:20.680076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:20.680076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:20.680076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:20.680076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 295/575] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:20.680076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.705239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78361-25a8-4937-ae33-d4fc7c9f9455 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.743633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.744528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.744829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.745105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.745916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.747340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.747656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.749325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.763744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 281/576] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.764405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.781987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:20.796123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108ac8a-2d70-4323-85be-28919aaf26d5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.834520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.835137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.835524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.835831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.836587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7c9ccc207322482e818b7c9f49aeed29) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:20.836899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:20.838343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:20.843721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 296/577] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/role_assignments?user.id=7c9ccc207322482e818b7c9f49aeed29&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:20.844521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.872758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b018dc0-524d-4f45-bd90-68b53bc4206b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.914610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:20.915647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:20.916118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:20.916676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:20.932200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:20.938386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:20.939361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:20.940158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:20.943974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 282/578] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:20.945293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:20.963423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e727c66-9834-475b-a89b-6b962d1051ee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:20.994220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.010054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.010670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.010947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.011215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.011859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.012159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.013578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.038006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 297/579] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.038852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.052093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.092896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8eaac-9643-45bb-9500-c4ef788ac957 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.138243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.139142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.139511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.139983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.140802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.141244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.142884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.154104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b806b-808a-4292-b9fc-c5bcd555ed7f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.159897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 283/580] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.159897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.193310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.194066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.194405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.194741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.201933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.203459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:21.216276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:21.217480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:21.218571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 298/581] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:21.219221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.265656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.266896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c36e-45ea-463f-9f61-f9e6ff96c184 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.298921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.305102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.305651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.305931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.306187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.306871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.307193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.308577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.318501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 284/582] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.319335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.334597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c757-558e-455a-8c06-0379fafae817 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.373105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.373744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.374017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.374284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.381973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.383428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:21.394450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:21.395379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 299/583] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:21.396283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.432758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3842-de8a-43a3-87d5-f93cee0a5497 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.467230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.467951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.468228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.468517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.475376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.476854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:21.488322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:21.489384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:21.490462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:21.496758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 285/584] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:21.496758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.510479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8d5cf-5374-492a-8ada-2b86b39eefd5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.517695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.547642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.548385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.548742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.549067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.549852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.550238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.551791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.555221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.570318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 300/585] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.575426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.606807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3a5aa-1cdb-4354-a452-ca63f92c1068 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.642420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.643184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.643466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.643801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.654174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.655686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:21.673513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:21.674667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:21.675723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:21.675723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:21.675723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:21.681354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 286/586] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:21.681354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.683921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefe1df-93f2-4016-9cfb-8fd6c9dc3939 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.724259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.725049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.725420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.725873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.726792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.727204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.728763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.751507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 301/587] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.751507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.774866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.788188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ecaea-28a7-46a6-ba6b-8609fe4dbd0f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.795713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:21.826216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.826738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.827017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.827289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.827986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:21.828293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.829775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.846605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 287/588] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:21.846605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.856255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676719d-fc71-43b2-be5c-4784e09a5fd0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:21.893503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:21.894654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:21.894934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:21.895202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:21.906488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:21.907978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:21.924396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:21.925500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:21.926386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:21.926386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:21.926386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:21.926386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 302/589] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:21.926629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:21.965929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9aff2-d503-411a-8942-1ee8c2b1b90e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.003096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.003734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.004013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.004344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.005031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.005340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.006739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.011978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.019690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 288/590] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:22.025604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.039647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72d25c-1c48-4614-b14a-9c9c089df606 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.053582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.074267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.075099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.075373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.075645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.086879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.088288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:22.102675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:22.103568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:22.104448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:22.104448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:22.104448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:22.104448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 303/591] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:22.105669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.134985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ccaf7-c7b3-40a5-bd91-1eea7baf3dd4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.170116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.171034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.171479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.171751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.187099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=cbe80d5de4bb4746a1756d1077e647a8, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.188545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.208777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 289/592] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:22 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:22.209762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.215017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c99a49-08d7-4476-a807-934f57a0e07a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.254978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.266857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.267481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.267756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.268022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.268899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.269130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.270508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.290325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 304/593] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:22.291053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.327862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.356276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d670a-09ce-4013-b3cf-1969414f316e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.398448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.399413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.399868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.400140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.407227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ced79-5b03-4b3f-b448-1750579b47c1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.407458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.408895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:22.422085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:22.423221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 290/594] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:22.428825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.450509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.451452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.451900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.452334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.453165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.453632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.455369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.470991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 305/595] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:22.471977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.526353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.535108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2f203-a385-464e-84d1-c6507fbf116d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.577022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.577743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.578088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.578551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.592925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.595037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.596559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.598502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496cbaa-348c-49ae-864d-bc1b9ccd5ff3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:22.623151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:22.624119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:22.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 291/596] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:22.625772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.640288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.641449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.641940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.642412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.643445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.643955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.645780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.664687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 306/597] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:22.665809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.752257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefa31-5d72-4861-b98e-0ea937561eb2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.771706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c17d-55fc-453f-9747-5e90cab54f10 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.775299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.795569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.796226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.796528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.796796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.803757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.805238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.811162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.811945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.812223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.812491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.813236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.813538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.814991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cff3da-4fb4-4d0e-89ff-1a762d35d169 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:22.816350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:22.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:22.818404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:22.822066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 292/598] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:22.822800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 307/599] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b1f9c-fbd0-4a26-a352-cc751e658e0e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b1f9c-fbd0-4a26-a352-cc751e658e0e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b1f9c-fbd0-4a26-a352-cc751e658e0e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 308/600] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:22.836673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b1f9c-fbd0-4a26-a352-cc751e658e0e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:22.887295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:22.937066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2851b-e42e-4c50-9c4b-807a38b7ab6b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.948902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b1f9c-fbd0-4a26-a352-cc751e658e0e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:22.971236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.971991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.972440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.972992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.983231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:22.983600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.983964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:22.984306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:22.984667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:22.985423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:22.985537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:22.985894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:22.987339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.004757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 309/601] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-be72841c-c616-4ae2-b44f-c10c6799c198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:23.004948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:23.006472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:23.007588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:23.007588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:23.007588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:23.007789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.007869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 293/602] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:23.007869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.007869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.007869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.023601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:23.024957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:23.033446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.096552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 294/603] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:23.096552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.115185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0bbc6-727e-4c4a-b7f5-0479ead5af31 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.149502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.153066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.153980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.154468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.154926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/32fb55fa700442dd974594ada94e7bb0/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.173820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=32fb55fa700442dd974594ada94e7bb0, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.175707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.197002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 310/604] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:22 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/32fb55fa700442dd974594ada94e7bb0/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:23.198534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.245320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd0652-4b55-4222-8163-1aef61bebfe2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.315459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.316290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.316584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.316905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.317570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:23.317867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.319283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.332536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 295/605] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:23.338336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.365952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.408299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b626e-ae3a-4085-9a2c-e23ce68b0cd9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.446955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.447799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.448379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.448761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/77f82ddcd0604a3697dc8b571719e1b6/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.467363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=77f82ddcd0604a3697dc8b571719e1b6, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.469422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.472127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf474602-f731-4bb8-b37c-6e27ea30b493 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.474608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.492607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 311/606] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:23 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/77f82ddcd0604a3697dc8b571719e1b6/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:23.494389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.535891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.537530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.537816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.538106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.561967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.563553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:23.590257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:23.591126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:23.592427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 296/607] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:23.598399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.656070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347100-a767-421a-abd7-79f3d077c0ea None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.687277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.699548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.700344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.700892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.701263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.714772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:23.720114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:23.721346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:23.722489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:23.725473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 312/608] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:23.730390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.794459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c2e28-e15f-4984-ae26-89a473a93d59 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.838621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.839346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.839724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.840000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.840737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:23.841087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.842728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.860653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 297/609] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:23.862086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.873216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b0d07-a853-4f45-b5d8-024d1224a269 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:23.896897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:23.910498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:23.911441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:23.911720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:23.911989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:23.912826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:23.913130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:23.914787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:23.925210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 313/610] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:23.933375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:23.999049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:24.048517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f61e6e-ddb9-4301-b88a-459f4a59ca14 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.087638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d26173-1888-4b88-99cd-33494d379665 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.089341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.090103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.090381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.090649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7f71b5efa95b41e196a505798527904d/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.106091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=7f71b5efa95b41e196a505798527904d, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.107592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.122172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 298/611] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:23 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7f71b5efa95b41e196a505798527904d/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:24.154468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.155141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.155506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.155772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.174949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.176525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-95e90e94-1196-4e75-862a-a70f1ff211f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:24.202181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:24.203512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:24.204385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:24.204385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:24.204385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:24.204385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 314/612] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:24.209575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:24.300745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:24.337444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb28e7-abb8-43ee-8693-65a629f1ed68 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.372554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.373178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.373454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.373719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.374375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:24.374671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.376071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.381747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f02a8-622b-4592-98ac-6d55bffdc91a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.382029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f02a8-622b-4592-98ac-6d55bffdc91a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.382293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f02a8-622b-4592-98ac-6d55bffdc91a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.384449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 315/613] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:24.390354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.390631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.390898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.391685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 299/614] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:24.397691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:24.412278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:24.413524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:24.485354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 316/615] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:24.506782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b74a6a-1ef6-4b26-a319-6f26bc627279 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.535428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:24.543392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.544370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.544721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.544999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.553138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.554534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:24.566162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:24.568279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:24.570515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:24.571961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 300/616] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:24.573954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:24.626094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:24.702547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca46a-e05d-4dc4-8354-cb19ef0f9164 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.736340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.737104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.737378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.737645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.738401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:24.738701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.740138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.758105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 317/617] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:24.763396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:24.901546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2fc04-ada2-4e67-98bf-3330bc21aa35 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:24.935347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:24.935986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:24.936266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:24.936555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:24.947517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:24.949014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:24.963407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:24.964461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:24.965710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:24.965710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:24.965710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:24.968613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 301/618] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:24.973625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:24.980375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.023860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.090006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b35df-ded7-4300-a2a9-dcd1015bd9d6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.111403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.137702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.138340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.138616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.138879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.141172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc1d5-eff8-4da9-9f86-9f8492bc0777 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.162195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:25.170562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:25.171493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:25.172452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:25.173127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 302/619] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:25.179209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.209869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.210486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.210763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.212165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.213020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:25.213324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.215052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.232807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 318/620] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:25.234734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.379934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.383963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ba71a-0eba-4658-a1ae-ebfb295c2f8b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.399217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723095a8-6039-4dbd-8b69-e21e023bd47f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.423339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.423872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.424151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.424878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.425595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:25.426170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.427732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77223fcf-a02a-491a-a742-bab037dfd3fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.438600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 303/621] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:25.439835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa69e-77dd-4a28-b3a1-6c79085be0cf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.439944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.440128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa69e-77dd-4a28-b3a1-6c79085be0cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.440388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa69e-77dd-4a28-b3a1-6c79085be0cf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.440588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.440818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.441144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.446092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 304/622] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:25.447209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa69e-77dd-4a28-b3a1-6c79085be0cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.452575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.454176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-31865719-1fe5-4d49-b8fa-70805bdd3239 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:25.469463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:25.470379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:25.471243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:25.475874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 319/623] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:25.475874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.475874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.475874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.493069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.502012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:25.503808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:25.570530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa69e-77dd-4a28-b3a1-6c79085be0cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.602171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 320/624] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:25.603312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.627711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.628639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.630455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.630866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.651088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.653411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:25.682486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:25.683637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:25.684729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:25.684729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:25.684729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:25.691158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 305/625] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:25.701666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.727459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.789738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b5f47-57bb-4760-8545-9b9b4d54f350 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.826940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.827558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.827837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.828114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.828841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:25.829148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.830601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759e02-b7c6-4930-9f75-71e237df884f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.831190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a9e5-f906-4d16-b055-c517634ef9ea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.833163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:25.843746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 321/626] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:25.845089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3e724-2bc5-4a60-9310-1e010fd2d362 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.845551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3e724-2bc5-4a60-9310-1e010fd2d362 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.845916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3e724-2bc5-4a60-9310-1e010fd2d362 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.847507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 322/627] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:25.850299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3e724-2bc5-4a60-9310-1e010fd2d362 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:25.872622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:25.873237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.873558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.873854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.874555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:25.874864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:25.876269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2e065-8741-4395-a734-ff2048a567dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:25.894133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 306/628] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:25.895464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:25.895773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:25.896057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:25.914872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:25.916116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:25.957916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3e724-2bc5-4a60-9310-1e010fd2d362 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:25.989807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 307/629] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:25.990674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.001805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.002803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.003849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.004438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.004967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7c9ccc207322482e818b7c9f49aeed29/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.024714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=7c9ccc207322482e818b7c9f49aeed29, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.026506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.045436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 323/630] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:25 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7c9ccc207322482e818b7c9f49aeed29/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:03:26.046610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.134423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.139954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58103c2-88ad-4c9f-8411-c2751f193a6a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.186673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.187717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.188167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.188657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.197750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.199590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:26.224080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:26.225677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:26.226885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 308/631] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:26.231226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.260472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31293985-68ff-49d6-a399-c9858dbaaba4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.297147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.298263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.299042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.299264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.300078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.312497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:26.318046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:26.318980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:26.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:26.327151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 324/632] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:26.328585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.342795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778834a-b671-4d4b-8b1f-dfa7dcc5af2a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.395182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.396045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.396325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.396623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.397614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:26.398301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.400110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.424251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 309/633] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:26.425143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.438673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c6970-d6d7-44de-81e6-7b5b75324e2f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.446994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.475786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.476408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.476795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.477126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.477867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:26.478236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.479716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.492867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 325/634] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:26.493857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.554931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ec3ab-fe89-4f1c-8165-8d7ad8280160 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.579099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.591433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.592446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.593193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.593531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.604023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:26.606110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf0f95-8cb8-4d08-91b9-ba2f04bc79de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:26.609018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:26.610103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 310/635] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:26.611110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.648844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.649459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.649733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.650005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.689913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.714457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1f07-46cb-4248-b23c-5c800cc05a86 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.729761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.731148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.750804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.751438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.751711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.751987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.762364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.763802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d79af4da-80ef-4ece-8ce5-11ad8e0f69ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:26.779139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:26.780097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 311/636] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b69-0f03-4070-bcfd-56b64c8a2dec None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b69-0f03-4070-bcfd-56b64c8a2dec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2b69-0f03-4070-bcfd-56b64c8a2dec None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.783531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 312/637] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:26.783974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d2b69-0f03-4070-bcfd-56b64c8a2dec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.822705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.846774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 326/638] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:26.847818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:26.899195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d2b69-0f03-4070-bcfd-56b64c8a2dec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.915161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:26.937318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.937917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.938197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.938465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.939132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:26.939432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.940860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d64cf-8e32-4c1d-8054-0a1b0a004ef3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:26.949148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 313/639] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:26.951004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.951430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.951849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.957395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467900e-3945-46be-92c5-b51260dea392 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:26.971223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:26.972467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:26.993585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:26.994126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:26.994405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:26.994671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:26.995314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:26.995612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:26.997034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.008770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 327/640] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:27.009539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.048250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 314/641] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:27.049142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.077012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.139271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39ed6d-771a-49c9-8fc7-fa7a79f7e7ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.158657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69dd7a-d8b7-4c6a-88fe-dd25761ee643 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.167466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.176675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.177312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.177590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.177855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.189154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.190719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.203838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.204770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.205263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.205752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:27.206631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.207506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:27.208349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:27.208349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:27.208349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:27.208349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 328/642] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.209005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.221393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.223860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:27.244696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.245726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:27.246584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:27.246584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:27.246584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:27.252803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 315/643] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.253836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.342160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.346167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3f43-7adb-4788-90e3-ac0b857aa7b8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.366707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d4f5-e940-4d4b-a921-8829e73d5b5c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.381581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.382177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.382480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.382710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.390269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.391713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:27.402956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.404044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 329/644] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.405672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.413853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.414468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.414749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.415012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.415660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:27.415968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.417823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.439311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 316/645] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:27.439311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.451060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.511407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264d55-fc17-4366-b580-f3d8b1eb2631 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.546343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41504249-ea0c-42d0-b638-4180440f7ab0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.550738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.551422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.551770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.552106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.552859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:27.553228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.555091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.575244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 330/646] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:27.575244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.580070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.580732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.581009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.583991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.592633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.596788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:27.603935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.605086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:27.606051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:27.609103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 317/647] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.609103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.683404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.687161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758bc3-3e9d-4eef-989d-c966dcd8ba66 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.723058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3f63-9be8-4e69-a7d5-492bc6d9997d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.724407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.725338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.725610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.726341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:27.726648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.728099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.750632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 331/648] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:27.750632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.763722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.764278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.764593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.764866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.771550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.773050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:27.784966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.786046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:27.787122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:27.790900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 318/649] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.791744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.852836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.855537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb55d5-79f1-49cf-91f0-b0959e3c2ac8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.892176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.893004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.893377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.893709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.897226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af095d8-2e5d-4770-8a48-424eb99a9715 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:27.901107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.902638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:27.914967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:27.916090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:03:27.917308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:27.917950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 332/650] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:27.923557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:27.939622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:27.940054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:27.941093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:27.941442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:27.941645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:27.942350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:27.942644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:27.944596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:27.960316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 319/651] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:27.961627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.029350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b7e8-0d5a-4d28-a78e-c70c818c4219 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.062710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42535-90ff-4c07-8c13-6de7f10592a5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.067042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.067657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.067932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.068208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.085048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.099859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.100463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.100763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.101032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.101720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:28.102017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.103445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.115264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 320/652] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:28.116082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.140819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.142291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.159854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.213060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0714-232c-4ab8-a5fb-95649e9cdfd8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.240532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 333/653] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:28.241367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.250612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.251579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.251861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.252138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.252910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:28.253217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.254764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a9af0-a867-4b0c-98ce-874b3bb54c1d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.271878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 321/654] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:28.272923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e1fe-44ec-4a65-b2ac-81ae99439ae9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.273207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e1fe-44ec-4a65-b2ac-81ae99439ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.273476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e1fe-44ec-4a65-b2ac-81ae99439ae9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.274662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 322/655] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:28.275348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28e1fe-44ec-4a65-b2ac-81ae99439ae9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.314920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.343138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e3626-afc3-440a-8b95-b221626139ee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.359729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.376554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.376746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28e1fe-44ec-4a65-b2ac-81ae99439ae9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.377191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.377463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.377710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.387744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.389239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:28.404078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:28.405174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:28.406038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:28.406038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:28.406038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:28.406038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 334/656] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:28.406038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.414436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.415039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.415317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.415582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.416289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:28.416625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.418060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.429884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 323/657] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:28.429884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.516280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15601b02-1fe9-4f90-8ef6-01ee9eddce7d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.551806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a55a14-4de8-4445-ae0f-476e03b17da2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.553592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.558481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.559126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.559386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.560150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.571618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.573154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-202c941d-26b2-444d-bf2b-1f01e3affbb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:28.589546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:28.590487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:28.591352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:28.591352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:28.591352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:28.591352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 335/658] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:28.591882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.592219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.592493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.599291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.599909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.600182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.600445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.611296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:28.612629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:28.633832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.682801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.684263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.699492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 336/659] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:28.700414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.807129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc432d2e-03e3-4a48-8566-d3da507b408d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.828681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 324/660] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:28.830103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.832738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.855312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.856539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.857019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.857835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.858955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:28.859689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.861964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.878100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 337/661] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:28.879873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:28.900738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:28.936769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8100e91-08ac-4027-b43e-ef042d56a930 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:28.974551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:28.975172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:28.975450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:28.975715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:28.976369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:28.976703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:28.978070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:28.990750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 325/662] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:28.991559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.003814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd190e-698a-49b8-ac27-40f7764f14d3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.040483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.041521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.042177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.042593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.053440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.056398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.057948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:29.072077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.073212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:29.074150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:29.074150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:29.074150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:29.078038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 338/663] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.078810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.095021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5b95-f554-444f-b0b4-5453d6ced8ab None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.129656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.130281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.130555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.130828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.137945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.139478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.146775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:29.152086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.153058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 326/664] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.153978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.193397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da67262-1fe3-4c80-a5d9-c6abdd1d6558 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.230460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.231464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.231760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.232030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.244767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:29.252097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.253056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:29.253990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:29.254651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e87b6b-c2fd-42cd-b74a-2a73415a0b46 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.256655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 339/665] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.257820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.278058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.290275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.290998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.291344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.291688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.299139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.300689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:29.312243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.313329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:29.314366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 327/666] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.314846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.367923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74c23f-1b96-448b-9668-c4d1b62a7e21 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.393150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.405097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.405955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.406228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.406621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.407257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:29.407830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.409247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.422943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a371789-cc7e-441f-95e9-ced9dda6e7dc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.426952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 340/667] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:29.427819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.457038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.457651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.457935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.458206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.458950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:29.459255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.460716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.477921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 328/668] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:29.478756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.528293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.540137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8b5f8-b565-41c0-97c9-7996a29d7832 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.578779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.579389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.579663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.579927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.580616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:29.580913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.582336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.590793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 341/669] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:29.591571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.602080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b5b4-8e9a-42c9-9c20-59a331f19971 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.638574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.640046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.640783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.641199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.641508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.642286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:29.642833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.644302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.662426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 329/670] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:29.662426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.698388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656735d-3772-404c-8e5f-8db45576941d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.732227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.732887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.733164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.733431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.740229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.741737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:29.753394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.754406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 342/671] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.755398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.766748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.770328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d272413-25eb-46da-9269-6dbcca43e4fa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.806335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.807022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.807358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.807684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.857279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98d14f-8e78-4a79-8fa0-4122affcc298 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:29.864346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:29.882062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.883840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.892259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:29.892862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:29.893131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:29.893394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:29.903824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:29.905298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:29.920335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:29.921384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:29.922243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:29.922243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:29.922243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:29.922243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 343/672] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:29.922647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:29.993570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.002953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 330/673] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:30.004008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.033693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ae9a4-bbc7-4c92-b805-afefc0a01eae None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.068457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.069087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.069359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.069635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.070338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=32fb55fa700442dd974594ada94e7bb0) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.070637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.072073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.078162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 344/674] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:29 2022] GET /identity/v3/role_assignments?user.id=32fb55fa700442dd974594ada94e7bb0&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.078989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.110846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.139413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c6d0-ee7f-4685-af1e-4ad5bc28068d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.188027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.189520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.190483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.191133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.192172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.192701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.195388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.213437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 331/675] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.214978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.216151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5298e-d470-4d1a-ab87-d7aeece9fbe9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.268117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.268757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.269095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.269362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.270000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.270300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.271715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.284480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 345/676] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.285248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.301663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.352230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db76aca-cf3d-4cad-aa8b-b573a31225f9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.390362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.391003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.391285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.391555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.399496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.402883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.404396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.413831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240b06f-ed6d-483f-af0f-d085ecea62d0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:30.418791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:30.420315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:30.421450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:30.421450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:30.421450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:30.421450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 332/677] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:30.421450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.458148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.458982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.459408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.459825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.461176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=77f82ddcd0604a3697dc8b571719e1b6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.461633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.463915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a4029-10a5-4e6c-b457-87e0bb712da9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 346/678] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/role_assignments?user.id=77f82ddcd0604a3697dc8b571719e1b6&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b817d-6cf1-49ae-8c92-e86d10e1cf33 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b817d-6cf1-49ae-8c92-e86d10e1cf33 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b817d-6cf1-49ae-8c92-e86d10e1cf33 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 347/679] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:30.481205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b817d-6cf1-49ae-8c92-e86d10e1cf33 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.537094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4583de-e3de-4cb2-9414-23a3384fb34e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.556214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.600868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.601956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.603129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.603610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.615949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.616859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b817d-6cf1-49ae-8c92-e86d10e1cf33 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.618017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6f45b64c-684a-4d40-9557-e08516aa57f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:30.637298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:30.638305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:30.639288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:30.641602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 333/680] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:30.643000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.643439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.643715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.662918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.664072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.664346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.664462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:30.664676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.665325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.665641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:30.665741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.667012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.682777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 348/681] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.683949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.694542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.755113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 334/682] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:30.756145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.814490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e81e-8273-4a75-b7a6-71c9ea50d2bc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.851582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.852226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.852498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.852841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.853574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:30.853916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.855514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.871495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c9c33-1d85-4862-8123-f8009d07a826 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:30.873176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:30.873766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 349/683] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:30.892470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:30.918235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:30.918872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:30.919491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:30.919760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:30.936098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:30.937587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:30.967044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:30.967996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:30.968911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 335/684] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:30.973694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.009990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.048811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0175e-9cb9-4656-8eea-1b1f8c6e875b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.084240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce1628-17a1-4981-a7b8-6871edc1137f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.097820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.098770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.099268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.099736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.120793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.121398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.121674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.121997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.122696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:31.123008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.124563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.137438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 336/685] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:31.138322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.166484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.199684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.201189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.248333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424e97-bdb6-48a6-868f-093326b2eccf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.261856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.286993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.287631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.287909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.288183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.300274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:31.305209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:31.306559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:31.307410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:31.308064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 350/686] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 28 10:03:31.308643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.311092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 337/687] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:31.312133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.402101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.413699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085414-65c9-4ec0-a533-d93b16765b7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.422409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b40164-59cb-498d-a96f-297065b0dd4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.449310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.449921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.450197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.450461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.451181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7f71b5efa95b41e196a505798527904d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:31.451477 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.453077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.458632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.459238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.459490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 351/688] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/role_assignments?user.id=7f71b5efa95b41e196a505798527904d&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:31.459658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.459820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.460261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.460437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:31.465006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.466426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.483293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 338/689] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:31.489148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.490208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.566023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5850cf-e4e5-475a-be5b-5e70ef64e5ce None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.605364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.606028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.606301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.606562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.639492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.641014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.651450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991777-20cf-4220-9eb5-a6b8ab942688 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:31.674453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:31.675540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:31.676416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:31.676416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:31.676416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:31.676416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 352/690] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:31.685842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.689275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.689934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.690213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.690486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.811864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.835247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8bba8-6eca-401a-a1cf-4a731140cc8e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:31.838683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.840158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.843634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:31.857471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:31.858634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:31.859726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:31.859726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:31.859726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:31.861445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 339/691] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:31.867235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:31.877012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:31.877626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:31.879159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:31.879159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:31.879388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:31.879655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:31.882409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:31.898099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 353/692] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:31.904020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.001512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a42-823a-4517-90f0-32ce73640d7b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.049373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650370-e96f-40a0-aa83-0fb2425cbb3d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.064388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.065021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.065295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.065553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.066259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:32.066558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.067973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.089443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 340/693] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:32.096226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.118004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.118678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.118960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.119235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.129658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.131129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.138989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:32.162877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:32.163804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:32.165374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 354/694] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:32.165844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.180209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.267192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a35-e4fc-47d4-aa82-3cbdd4b60e5b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.337902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ebc88-aae3-40db-8bb0-4f6b2b709960 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.346790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.347840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.348296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.348716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.369651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.371116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.377629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.378643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.379109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.379671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.380735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:32.381248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.383475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.397026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 355/695] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:32.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:32.403079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:32.404024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 341/696] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:32.405519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.408809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.478272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.530704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b231cf-5ea2-4573-a839-807553d29c37 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.534543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef603b-8a19-420e-a3c4-1b12b8b453c2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.548728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.568830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.569445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.569777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.570042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.570816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:32.571117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.572663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300136c-d7fa-45df-8efa-75b8fcd9f774 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.579752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.580843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.584244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.584757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.585716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7c9ccc207322482e818b7c9f49aeed29) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:32.586213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.588637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc5e71-84f0-4c54-bce6-4676f7098107 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.594976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 342/697] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:32.596412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd3ba-0ef5-4d2c-bea7-f196aba6fb0c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.596719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd3ba-0ef5-4d2c-bea7-f196aba6fb0c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.597066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd3ba-0ef5-4d2c-bea7-f196aba6fb0c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.604533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 343/699] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:32.604533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cd3ba-0ef5-4d2c-bea7-f196aba6fb0c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:32.604922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 356/699] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:32 2022] GET /identity/v3/role_assignments?user.id=7c9ccc207322482e818b7c9f49aeed29&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:32.606536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240559-8f15-4b82-b6db-23007a7f116a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.607287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240559-8f15-4b82-b6db-23007a7f116a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.607746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240559-8f15-4b82-b6db-23007a7f116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.644478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-84240559-8f15-4b82-b6db-23007a7f116a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:32.645726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-84240559-8f15-4b82-b6db-23007a7f116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:32.748145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 357/700] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:32.750240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ce1db-0d19-452b-a734-753a468bf770 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.750531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ce1db-0d19-452b-a734-753a468bf770 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.750818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ce1db-0d19-452b-a734-753a468bf770 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.756996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 358/701] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:32.758594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.758866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.759147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.764601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cd3ba-0ef5-4d2c-bea7-f196aba6fb0c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:32.784978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:32.786806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:32.805836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:32.806854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:32.807336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:32.807802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:32.852080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.864913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:32.874921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 359/702] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:32.892893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:32.894390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:32.995789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 344/703] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:33.001156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.072222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.130851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203ca4-13af-4ede-a143-d22e4cb6dea9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.169557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.185579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.186811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.187103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.187381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.198323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.199918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.205009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf28d5-d82b-48b9-8819-3ab429fe8ee8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:33.215593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:33.216754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:33.217919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:33.217919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:33.217919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:33.217919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 360/704] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:33.217919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.249377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.250666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.251373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.251801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.261212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.269548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.271028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:33.298129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:33.299147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:33.300022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:33.300022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:33.300022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:33.307480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 345/705] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:33.307480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.351631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d6ba-d0b4-41a7-aa4d-1d4123e7bf30 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.393764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.394428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.394710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.394980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.395584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.397013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:33.408491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:33.409571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:33.410388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:33.414593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 361/706] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:33.415481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.425571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1ed3d-bd21-4230-9c5a-c479fa861029 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.461955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.474096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.474718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.474998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.475321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.475982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:33.476316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.478437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.498349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 346/707] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:33.499371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.532015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.537771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01375a-7703-4a0d-8dfc-c6a1e0de0286 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.577038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.578116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.578671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.579143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.580028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:33.580560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.582260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.603036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 362/708] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:33.604138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.625009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46120-3a55-4afd-b70b-09d0e7fcf8f6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.662119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.662651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.662929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.663198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.664094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:33.664394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.666117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.676445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 347/709] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:33.676445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.738227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.739767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2cbcf-d887-40b6-b961-037735892d6d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.788051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.789286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.789996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.790473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.793657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded3e0-fa26-4b33-83b4-b18d2807e8ca None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.801542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.803747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:33.819911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:33.820887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:33.821791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:33.824574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 363/710] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:33.826390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.840574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.841193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.841473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.841747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.842402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.843850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:33.852530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:33.867622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 348/711] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:03:33 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:33.867622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:33.951802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e5b1-ad37-49d6-a35f-c21858a73954 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:33.994583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:33.995404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:33.995756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:33.996093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:33.996852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:33.997297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:33.999020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8bdc-6e58-423f-b363-069f676242dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:34.011456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 364/712] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:34.013053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376124ad-a33f-4a2f-bd0f-fd9af6dd086f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:34.013269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafce500-0354-4a9d-a816-48e674f1159a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.013626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafce500-0354-4a9d-a816-48e674f1159a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.013968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafce500-0354-4a9d-a816-48e674f1159a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.015937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 365/713] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:34.022103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.022512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.022951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.027128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:34.054334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:34.056112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:34.066874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:34.067514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.067790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.068063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.069178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:34.069499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:34.071298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:34.087808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 349/714] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:34.097983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:34.166670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 366/715] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:34.225867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be925b-d43d-43f4-95e5-fa1dacb889a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:34.263272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:34.264242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.264756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.265227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.266171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:34.266695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:34.268388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:34.276718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 350/716] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:34 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:34.445374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:34.462656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:34.577465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8cf2b-cc1b-484d-84d7-0e64d9b921c3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:34.613497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:34.614149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.614428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.614704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.615295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:34.616715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:34.626091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:34.627085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:34.627969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 367/717] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:34 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:34.633795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:34.703844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:34.805485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dfd8-ad57-4509-8f9f-455bf0955d5b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:34.855597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:34.856610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.857092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.857579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.858412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:34.858924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:34.860518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240858c4-99a3-4717-bc35-d62120d86d4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:34.868355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:34.873793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:34.939861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c9f7-ec37-4f9a-90ca-95a315ef5fd3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.940227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c9f7-ec37-4f9a-90ca-95a315ef5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.940408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69c9f7-ec37-4f9a-90ca-95a315ef5fd3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.942060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 352/719] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:34.947513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:34.947799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:34.948075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:34.977396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:34.978590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:34.979108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:34.991277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1aa505-2bc2-4802-8f47-b6c471e33f3b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:35.031864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:35.032525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:35.032864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:35.033143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:35.033946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:35.034259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:35.036057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:35.041871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 368/720] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:35.119852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 353/721] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:35.447791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:35.556809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4640b01-1925-488b-b12a-16055f3289d7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:35.592537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:35.592537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:35.592537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:35.592537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:35.602682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:35.608003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:35.608933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:35.609743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 369/722] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:35.621638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:35.672779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:35.772006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7baee-4292-4cd6-8365-2e8674518139 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:35.817984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:35.818607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:35.818885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:35.819158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:35.819842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:35.820146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:35.821568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:35.830741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 354/723] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:35.836308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:35.843097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:35.971205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba58f9-ef1a-4101-9ea4-26393e89854a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:36.005926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:36.006567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.006849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.007122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.018611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:36.020154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:36.035781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:36.036753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:36.037616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:36.037616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:36.037616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:36.037616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 370/724] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:36.049656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:36.158917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:36.175958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e50418-8790-4109-9994-9e2b9f423660 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:36.220235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:36.221169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.221561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.221939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.222947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:36.223394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:36.225519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:36.252646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 355/725] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:36.254785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:36.330242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:36.375699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0e42-b930-4f1f-bbbc-a869248677eb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:36.409296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:36.409966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.410242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.410508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.419596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:36.421097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b3df99c7-3e0f-42fe-8f71-89bbcac5abce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:36.432772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:36.433705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:36.434614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:36.438654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 371/726] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:36.440214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:36.501997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a6ab-2cf9-4662-a102-47d0ddc04cd5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.502550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a6ab-2cf9-4662-a102-47d0ddc04cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.502550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a6ab-2cf9-4662-a102-47d0ddc04cd5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.504205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 372/727] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:36.508905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dbbd-d171-40b0-9c97-3085ac6c0d54 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.509186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dbbd-d171-40b0-9c97-3085ac6c0d54 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.509453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8dbbd-d171-40b0-9c97-3085ac6c0d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.541541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-88d8dbbd-d171-40b0-9c97-3085ac6c0d54 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:36.542752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-88d8dbbd-d171-40b0-9c97-3085ac6c0d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:36.560055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ae1b1-1520-4526-a1cf-7f4677cffabc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:36.606688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:36.607654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.607936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.608207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.608965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:36.609266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:36.610872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b39c9-1141-418e-b147-468ed3956b61 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:36.636157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 356/728] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:36.637872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2431-a040-4c20-9e02-bdcc0876e8aa None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.638166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2431-a040-4c20-9e02-bdcc0876e8aa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.638440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2431-a040-4c20-9e02-bdcc0876e8aa None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.640325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 357/730] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:36.640564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 373/730] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:36.641143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2431-a040-4c20-9e02-bdcc0876e8aa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:36.645209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de4b5-b548-4972-93b8-6f11393fc82a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.645492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de4b5-b548-4972-93b8-6f11393fc82a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.645759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06de4b5-b548-4972-93b8-6f11393fc82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.646321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:36.667262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a06de4b5-b548-4972-93b8-6f11393fc82a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:36.668919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a06de4b5-b548-4972-93b8-6f11393fc82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:36.737644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:36.740482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 374/731] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:36.742436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2c92d-47ab-4f89-8e80-124cd2769808 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.743012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2c92d-47ab-4f89-8e80-124cd2769808 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.743553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2c92d-47ab-4f89-8e80-124cd2769808 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.745312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 375/732] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:36.751792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.754840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.755340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.787153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:36.790002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:36.813701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2431-a040-4c20-9e02-bdcc0876e8aa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:36.875868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:36.876870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:36.877369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:36.877835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:36.918922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 376/733] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 28 10:03:36.920453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:36.999837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.002497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.016826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.088085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28041fe-bd19-4130-90d5-ee6c14655011 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.090721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.133205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 358/734] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 28 10:03:37.133696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.137273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.138790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.139076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.139347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.140014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.141520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.168571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 377/735] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:03:36 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:37.168571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.280030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b90a-a32a-4adb-b9a2-de1dfe742b21 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.282794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.321822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50970-3851-419c-b32a-44829b8ee389 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.328655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.329412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.329771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.329938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.342988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:37.347988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:37.348962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:37.350206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 359/736] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:37.350958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.365309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.366095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.366501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.366819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.378953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.380458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:37.396777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:37.397744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:37.398816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:37.398816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:37.398816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:37.398816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 378/737] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:37.399046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.421507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.463179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b95b-7996-469a-bc07-9e32e670754e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.497756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.498390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.498667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.498935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.509891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:37.514756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:37.515683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:37.516615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:37.520718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 360/738] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:37.521619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.523963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897c02a-77f5-4513-8aec-3a72ef2ffc45 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.552426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.570970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.571599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.571880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.572151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.573311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:37.573615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.575721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.596616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 379/739] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:37.596616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.641320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a562f-d44f-4ce1-8c14-4d60aa95fbdd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.679568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.680663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.681009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.681352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.682004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:37.682304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.683678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.685854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.700559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 361/740] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:37.700559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.712926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa1d8d-3eb1-4ed9-97a2-b9fea38a4c3c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.747084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.747699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.747975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.748248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.748951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:37.749256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.750648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc00f1-d163-4b19-932e-e1b2b90d82a0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.760705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 380/741] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:37.761929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fc700-c745-4831-980e-3a6ad979da06 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.762210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fc700-c745-4831-980e-3a6ad979da06 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.762479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fc700-c745-4831-980e-3a6ad979da06 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.763824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 381/742] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:37.765227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fc700-c745-4831-980e-3a6ad979da06 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.799966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.816390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70a42-bf6d-4e85-8a60-8f73273645f3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.850276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.850911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.851191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.851469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.863183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.864798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.878858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fc700-c745-4831-980e-3a6ad979da06 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:37.879726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:37.880696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:37.881594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:37.881594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:37.881594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:37.881594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 362/743] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:37.881810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:37.918586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:37.919557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.919826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.920088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.922832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:37.927503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:37.929095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4eb76769-a05a-42f3-b5fa-d0b1002aa046 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:37.943592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:37.944705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:37.945620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 382/744] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:37.946263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:37.946577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:37.946855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:37.965303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:37.966512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:37.995557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfac2d0-2e47-4c19-9ea4-e6337738c49f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.032268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.032946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.033229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.033504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.036459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.037343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 383/745] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 28 10:03:38.038243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.057724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.059583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:38.075005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:38.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:38.077263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:38.077263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:38.077263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:38.083709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 363/746] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:38.084733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.168061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03db3ad-b64a-40fc-aa4a-daa3bc40a2b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.191329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d37c-4f99-4253-a615-df43faec72cd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.201997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.213253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.213990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.214365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.214729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.216017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:38.216416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.218445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.230308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.231119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.231544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.231958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.232836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:38.232950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 384/747] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:38.233377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.233949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.235314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.249058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 364/748] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:38.250271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.318292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.356203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313372a-91fc-4692-9ed6-12de3191866e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.378284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cb005-c0a3-4d27-b0ac-6bfdb355db39 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.391132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.391766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.392049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.392294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.392992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:38.393302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.394881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.408215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 385/749] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:38.409601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.424221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.425322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.425894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.426167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.434035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.435729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:38.448357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:38.449316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:38.450244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 365/750] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:38.450824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.461429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.519566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c97d0-54cf-410e-b5e1-228b826572f0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.558103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fab345-21e2-4897-be63-8f7f86454d94 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.559519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.560285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.561392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.561669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.562402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7c9ccc207322482e818b7c9f49aeed29) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:38.562708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.564147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.567185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.576701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 386/751] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/role_assignments?user.id=7c9ccc207322482e818b7c9f49aeed29&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:38.581954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.603288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.603956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.604241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.604527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.605331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.606806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.628089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 366/752] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:03:38 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:38.629270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.714370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cd4fc-04be-4f1f-964f-a9e727f63534 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.733490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.767401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca9d2-4ce0-4270-b796-ce948f6f80c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:38.788964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.789715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.790056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.790384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.807942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.809987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.819930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:38.820806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:38.821153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:38.825183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:38.826037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:38.826700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:38.829142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:38.837178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:38.838173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:38.839085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 387/753] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:38.848255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.872174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 367/754] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:38.880533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:38.920700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:38.983608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932af2cd-d898-4566-9d41-4c3f235633e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.023704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.024432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.024762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.025038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.025831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:39.026141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.027303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950289c-0712-413a-bbff-224114cc8708 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.027642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.047355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 388/755] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:39.052874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.069292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.069916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.070192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.070512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.099155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.196288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0280d-4d3f-4cd3-9e97-f283f39d6f1a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.226923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.228515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b176c-f124-473e-bbb5-0e06e972bdc9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.231126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.231929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.232344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.232807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.243765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.245783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1e1d9473-bae9-488e-9eb7-6a8f29d4c719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:39.266441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:39.267437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:39.268285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:39.268285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:39.268285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:39.268285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 389/756] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:39.269575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b6fb-0f3b-4ce5-bc7a-d38f67ca0555 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.269856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b6fb-0f3b-4ce5-bc7a-d38f67ca0555 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.270123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b6fb-0f3b-4ce5-bc7a-d38f67ca0555 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.275727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 390/757] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:39.275865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.275972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b6fb-0f3b-4ce5-bc7a-d38f67ca0555 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.368752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 368/758] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:03:39.370449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.370925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.371383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.384768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.394412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:39.397041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:39.410613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b6fb-0f3b-4ce5-bc7a-d38f67ca0555 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.454810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.455575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.456121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.456448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.457394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:39.457699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.459244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.474319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 391/759] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:39.475574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.486693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 369/760] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:39.487625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.539988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.583346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edba5b1-40d6-45c0-b844-fd6e8590861e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.601686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8fad2-51a9-484a-884e-5fcdd2b10bea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.621640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.622283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.622559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.622826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.643255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.647392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.648905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:39.670515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:39.671474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:39.672787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:39.672787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:39.672787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:39.672787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 392/761] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:39.678799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.681233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.681895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.682176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.682448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.701581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.703056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:39.729720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:39.730639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:39.732089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 370/762] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:39.737383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.826197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8781d94-3b30-4b6c-9fce-12ea312a3bf9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.872283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.873279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.873797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.874730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.875843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:39.876347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.878852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.886442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868c1d-0831-4e7a-af92-4791aa78a97a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:39.900897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 393/763] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:39.900897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.903921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:39.928164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:39.928862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:39.929157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:39.929681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:39.930205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:39.930513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:39.932141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:39.946705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 371/764] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:39.947596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:39.975616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.046215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660705-1a2c-4d1c-82d4-d5667340434a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.062739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad980-5d52-4bb6-b423-f380b58448f9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.098909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.099091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.099457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.099734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.099999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.100954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.101427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.102177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.103330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.104985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.106881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.108495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:40.116324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:40.117400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:40.118216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:40.120250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:40.121288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:40.122383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:40.123444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 394/765] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:40.127826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 372/766] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:40.127947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.128889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.162036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.237022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea4c6a-0f78-435d-9067-a45159729c28 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.266518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3e68-9d66-4dfa-a3e7-feab2ffeb5f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.273945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.274613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.274892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.275161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.275796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:40.276093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.277499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.282484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 373/767] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:40.283331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.289646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.314437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.315162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.315439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.315815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.318463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=cbe80d5de4bb4746a1756d1077e647a8, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.320373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.352614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 395/768] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:40 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:40.352614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.458825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.470161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f730-d80c-42b5-9184-6d1b71db75e8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.528177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.528870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.529150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.529421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.530134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:40.530440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.531897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.553323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 374/769] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:40.557068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.560827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc434c77-83b8-4b86-961b-72b9de3e896b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.612694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.613355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.613673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.613882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.614522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:40.614826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.616227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.624384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 396/770] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:40.636562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.694363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4522-eb02-4f72-95bc-ea69d93de277 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.720863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.730420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.731037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.731361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.731640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.732389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=207eced18c9b455186567c94927a9754) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:40.732747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.734554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e444b-6534-4292-b871-826c96fa957f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.741737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 375/771] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/role_assignments?user.id=207eced18c9b455186567c94927a9754&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:40.790047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16b548-2fdc-40b8-b9af-f046a97c6947 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.790407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16b548-2fdc-40b8-b9af-f046a97c6947 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.790602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16b548-2fdc-40b8-b9af-f046a97c6947 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.792322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 376/772] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:40.797893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a61c1-0f3c-4573-b99b-db55df65afbf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.798176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a61c1-0f3c-4573-b99b-db55df65afbf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.798453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a61c1-0f3c-4573-b99b-db55df65afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.807653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa9c60-1314-4d82-b2c6-26b71e1cf969 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:40.837815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c78a61c1-0f3c-4573-b99b-db55df65afbf None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:40.839062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c78a61c1-0f3c-4573-b99b-db55df65afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:40.853291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:40.854323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.854803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.855480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.856251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:40.857125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] RBAC: Adding query filter params (interface=public, service_id=636be03928424299a98259494bd7f4ae, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:40.857125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:40.858551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:40.879308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 397/773] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/endpoints?service_id=636be03928424299a98259494bd7f4ae&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:40.889455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:40.942858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 377/774] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:40.944284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c430-f0a5-4b98-843d-7b1bb19204fb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.944602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c430-f0a5-4b98-843d-7b1bb19204fb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.944875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c430-f0a5-4b98-843d-7b1bb19204fb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.946342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 378/775] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:40.951404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:40.951684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:40.951952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:40.979306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:40.980514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:41.095901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318e30a-e53d-49a8-99ae-a430e11da3e3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:41.098781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:41.131877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:41.132872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:41.133359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:41.133870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:41.134666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:41.136234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:41.142027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 398/776] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:03:40 2022] GET /identity/v3/services => generated 772 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:41.152681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 379/777] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:41.421670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:41.523865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00b420-f4fe-40aa-869a-4b2dacc39833 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:41.559313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:41.560312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:41.560814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:41.561296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:41.572020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:41.573193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:41.577489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:41.578377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:41.579172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:41.581903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 399/778] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:41.587948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:41.617677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:41.690530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ffb5c-9339-431d-8d22-30e1fbb883c9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:41.724356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:41.725014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:41.725993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:41.726260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:41.726921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:41.727221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:41.728620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:41.745256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 400/779] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:41.764532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:41.790990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:41.830722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679271b3-5a3d-4f1c-bb2c-1c72311611d9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:41.898047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:41.898689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:41.899012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:41.899283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] PATH_INFO: `/v3/services/network` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:41.899889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:41.901278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:41.924717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:41.925777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:41.926611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:41.928614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 380/780] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:41 2022] GET /identity/v3/services/network => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:41.928789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:41.934864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:41.942729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae101-8f48-4862-b343-c7636f5df218 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:41.993576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:41.994368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:41.994741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:41.995123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.006252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.007838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:42.022605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:42.023838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:42.025032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:42.025032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:42.025032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:42.027451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 401/781] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:42.036453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.160724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9d49a-b84e-4f7e-8874-a4fe7bb912c9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.201396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.203055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.203336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.203612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.204254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:42.205657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b4c01-9901-4a8d-a7c6-ed225ad4147c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.207405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.208842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.237239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 402/782] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:42.243740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.253479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:42.259624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.260246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.260928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.260928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.261499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:42.261816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.263436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.269148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 381/783] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:42.277209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.296976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:42.409274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da5f3a-d0d1-42df-ae45-51c4d0784690 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.453138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.453671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.453947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.454212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.470055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.471532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:42.499446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:42.501282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:42.503299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 403/784] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:42.510506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfbef-ad2f-4159-a0a4-b7321b41cea4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.513395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.591681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.592310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.592609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.592877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.593639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:42.593939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.596476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.613426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 382/785] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:42.617210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.635621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:42.697075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:42.732642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336390f-d064-4a16-85ee-2de8cc72b58b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.758419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d3a38-b55d-4ce6-9bc0-43661224be45 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:42.769688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.770316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.770597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.770856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.772322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] RBAC: Adding query filter params (interface=public, service_id=c8c4951f8754467ab94b6ca8f849740d, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:42.772754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.774229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.784809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 383/786] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/endpoints?service_id=c8c4951f8754467ab94b6ca8f849740d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:42.792407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.838970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:42.841239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:42.841523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:42.841788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:42.842777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:42.843078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:42.844543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:42.881563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 404/787] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:42.887361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:42.963575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75224c-4a45-475a-932c-5870165ffd10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.010531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.011378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.012229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.012537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.013172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.015444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23492b-efe6-47eb-80e8-bd733a500e48 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.018045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:43.022679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 384/788] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:43.024253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e936-822b-4eee-9bc2-9c35ab6d5e74 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.025027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e936-822b-4eee-9bc2-9c35ab6d5e74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.025290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e936-822b-4eee-9bc2-9c35ab6d5e74 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.026985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 385/789] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:43.035570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.035570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.035570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.046427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e8c6c-ebe9-4ccf-9b3d-74856fe098cb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.059343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:43.060840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:43.072895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:43.083904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.084598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.084881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.085151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.096430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.102274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:43.137954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:43.138949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:43.139831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:43.139831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:43.139831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:43.141599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 405/790] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:43.149621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.167690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 386/791] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:43.307923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297dd3f-4457-4a8d-94b8-46cf11f0351d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.341630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.342250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.342527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.342793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.343431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:43.343729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.345144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1ffe3-35be-4dc8-89be-22692d07135e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.357532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 406/792] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:43.363079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.381003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f648-bcaf-4ffd-9706-39b14d73812b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.381556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f648-bcaf-4ffd-9706-39b14d73812b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.382040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61f648-bcaf-4ffd-9706-39b14d73812b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.383735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 407/793] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:43.389713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.390000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.390262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.414045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:43.415390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:43.457830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:43.512007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b8673-4aa5-4aa7-8be1-bd8a37e3168e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.542874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 408/794] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:43.544071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.556369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:43.572358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.573119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.573326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.573605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.583063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.590810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:43.620801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:43.621749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:43.622619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 387/795] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:43.627532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.657129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a6a1a-032c-4259-91fa-450b1efbbac7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.695877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.696909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.699287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.700976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.721926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:43.728188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:43.729119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:43.729946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:43.734397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 409/796] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:43.740594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.772389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:43.811815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1b18a-5b9e-41a7-87fe-dcb707e69647 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.860970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10028cc6-6d21-451f-a190-cb3a49e9af49 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:43.900246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.900891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.901168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.901440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.902119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:43.902421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.903821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.905664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:43.906281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:43.906555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:43.906824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:43.907527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:43.907828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:43.909285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:43.925038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 410/797] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:43.926207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.927068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 388/798] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:43.936307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:43.997155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.025608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.044989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e4e61-f374-4b94-ba47-4ef5b79393cd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.066332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cccad-de0d-4c2f-80d0-c913b2e6fda7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.081882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.082523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.082801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.083070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.093219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:44.098606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:44.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:44.100781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:44.104640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 411/799] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:44.105417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.106163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.106800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.107076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.107345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.119500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.121259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7b711f4d-fd2c-4451-868c-6a7a48d00307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:44.136181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:44.137161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:44.138049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:44.138049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:44.138049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:44.138049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 389/800] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:44.138789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b943-d21c-4292-8abc-e2edc168375e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.138789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b943-d21c-4292-8abc-e2edc168375e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.139041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b943-d21c-4292-8abc-e2edc168375e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.140367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 390/801] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:44.140921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859b943-d21c-4292-8abc-e2edc168375e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.224557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0baf7-1ec8-416f-9993-b1a58ccd2f72 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.245269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859b943-d21c-4292-8abc-e2edc168375e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.246795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.261522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.262496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.263013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.263448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7c9ccc207322482e818b7c9f49aeed29/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.265734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=7c9ccc207322482e818b7c9f49aeed29, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.267253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.280162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.280797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.292953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 412/802] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:44 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/7c9ccc207322482e818b7c9f49aeed29/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:44.293172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.293172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.293333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.293414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:44.293414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.293414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8a674-671d-46a7-bb37-edd5fa11f4d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.297294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.307459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 391/803] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:44.308939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.309305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.309636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.332546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:44.333942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:44.419969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d1cbd-7b06-463c-8e8a-f35d8fd5364d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.441470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 392/804] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:44.442470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.460552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.461618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.462188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.462673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.463509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:44.464002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.465794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.479901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 413/805] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:44.485474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.567678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.571407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.581797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0b60f-f9b9-4232-b1cd-03e5e662f415 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.620046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b80b-b045-49f0-bf1d-bd0b1cff8891 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.625046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.625741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.626023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.626296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.640891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.642359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:44.661518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:44.662448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:44.663327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:44.663327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:44.663327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:44.663327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 393/806] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:44.669781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.679061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.680764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.681246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.681990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.692210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.697850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:44.724926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:44.725921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:44.726838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 414/807] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:44.734860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.808289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476c37-dfff-4775-9b88-c06d11037567 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.852866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.853784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.854064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.854334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.855716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:44.856042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.857707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.858521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9d160-1413-4c52-8c93-831488db2207 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:44.872142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 394/808] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:44.873017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:44.882181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.892244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:44.905717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:44.906556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:44.907125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:44.907624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:44.908626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:44.909086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:44.910727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:44.932825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 415/809] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:44.933977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.001100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3d96c-0f16-4172-b404-2c7e99c8fabc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.043958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fddd0-2918-4dd7-9e88-7a32a1784d67 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.045167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.045796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.046076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.046347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.047179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.049519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:45.060630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:45.061605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:45.062425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 395/810] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:45.063232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.088332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.089360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.089877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.090389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.099030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.101068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:45.114177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:45.115128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:45.116076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:45.123308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 416/811] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:03:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:45.124533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.144769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.175233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.186683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecf4a-11c0-4fce-b882-16a2e3b03d8c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.232809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.233440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.233723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.233993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.239709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8db154-4c9b-4d54-afb6-f52cc291777c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.276605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.277688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.278185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.278664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.279488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:45.279978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.281724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.287348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 417/812] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:45.288426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.320941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.322870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.392359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb3b96-3d35-4097-91cd-ee8208988373 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.399760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.408246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.427214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.427829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.428106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.428374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.429130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:45.429434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.430891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb496ac-8da5-4eb8-a202-2ac96f69e963 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.445102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 396/813] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:03:45.445929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.453908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 418/814] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:45.455900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef611-a172-4bf8-99af-3a6ec241f3a2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.456862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef611-a172-4bf8-99af-3a6ec241f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.457358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef611-a172-4bf8-99af-3a6ec241f3a2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.459392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 419/815] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:45.460470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef611-a172-4bf8-99af-3a6ec241f3a2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.548771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cbf49-cf66-48ec-94f1-e1ca39632995 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.589713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef611-a172-4bf8-99af-3a6ec241f3a2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.591565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.592876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.593163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.593439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.594092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:45.594395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.595769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.605973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 397/816] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:45.605973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.641743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.716225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f876cc-52d1-4f23-a5d2-5bd7c079fc80 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.745559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.746221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.746514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.746714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.751722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.752344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.752655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.752926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.757135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.758596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.763853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.765303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-68d664e1-c24f-4558-9827-b194f4270a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:45.774806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:45.775751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:45.777120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:45.777120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:45.777120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:45.777120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 420/817] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:45.777120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.777416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.777659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:45.785751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:45.786915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:45.787935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:45.787935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:45.787935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:45.791429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 398/818] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:45.792255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.799920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:45.801187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:45.813644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.877891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 421/819] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:45.879354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:45.889707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:45.896791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b778ad-f79b-4381-a0e0-09eb26591001 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:45.933343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:45.934072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:45.934422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:45.934761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:45.935564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:45.937340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:45.964781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 399/820] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:03:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:45.969616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.038146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4c94a-3d45-4a66-80de-53a7a350bc64 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.123736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.124365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.124668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.124934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.125596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.125899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.127324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.156284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 422/821] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.161325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.161473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2073b6-406f-48ae-af1b-522be0c6642f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.166685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.208108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.209949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.209949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.209949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.209949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.210243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.211596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.226475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 400/822] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.227242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.231666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.295844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6784543-d707-4a47-8a71-13edbaf91d0e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.330418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.331403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.331772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.332140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.339721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.341328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.348759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51625d3-d008-4366-a2f9-c68c4b2ede9d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:46.353926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:46.354997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:46.356088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:46.357902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 423/823] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:46.358863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.389429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.390406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.390858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.391278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] PATH_INFO: `/v3/services/network` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.392115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.393593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:46.402634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:46.403766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:46.404951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 401/824] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:46.405917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.428389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.469154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c300d21-dd49-420f-a54a-8ae8c7cb41cc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.474022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.504730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.505274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.505544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.505804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.511880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299722-0c51-485b-8f2a-7200627fa661 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.512403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.513893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7341b4f1-06d0-40b2-80e7-f1e0ccf40361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:46.525344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:46.526413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 424/825] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6ea9-ed49-459a-9531-86de1157ed1f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.531838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6ea9-ed49-459a-9531-86de1157ed1f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.532455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6ea9-ed49-459a-9531-86de1157ed1f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.534032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 425/826] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:46.534919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6ea9-ed49-459a-9531-86de1157ed1f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.556713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.557329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.557609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.557885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.558581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.558885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.560313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.572392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 402/827] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.573231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.650048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6ea9-ed49-459a-9531-86de1157ed1f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.673759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.678898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9252c8-7afa-49b1-be79-e458b3a397d6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.686613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.687531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.687943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.688351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.689184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] RBAC: Adding query filter params (name=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.689621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.691272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04e9db-7eba-407b-a1b1-c25766dc3613 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.700922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 426/828] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.702373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.702799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.703201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.706292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.716599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.717222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.717499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.717824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.718506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.718807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.720232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.722167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:46.723448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:46.736689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 403/829] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.736689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.800638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 427/830] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:46.800638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.847634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b36b-0b9f-4bcb-8f35-99f2e527def3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.883785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.884390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.884694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.884962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.885695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.885996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.887548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.894345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 404/831] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.895156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:46.918606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18573c35-8899-467d-b1c8-786cd3170aaf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:46.945558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.954055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:46.971799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:46.973159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:46.974222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:46.974694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:46.975831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:46.976330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:46.979132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:46.985914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 428/832] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:03:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:46.994947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.041707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f4e2-c54f-4c8f-aa22-3f751c89f492 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.110944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.111667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.112895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.113175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/207eced18c9b455186567c94927a9754/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.135486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=207eced18c9b455186567c94927a9754, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:47.137021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:47.162694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 405/833] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:03:46 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/207eced18c9b455186567c94927a9754/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:03:47.163585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.178924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de221ad3-2802-4446-b343-07c4faeb3dd7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.222517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.224686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.225346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.225859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.226835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:47.228829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:47.261749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 429/834] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:03:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:47.290946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:47.295529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc114a5-f7be-46f0-bbdb-6f8a8c1e8191 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.343336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.344411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.344913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.345402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.358413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:47.363986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:47.365004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:03:47.365815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:47.367970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 406/835] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:47.374405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.415126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:47.492857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42d3dc-42b0-4ce9-9466-05611aa184a6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.529199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.530104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.530533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.530941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.531749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:47.532187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:47.533750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:47.543174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 430/836] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:47.549007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.669922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23afd1f-3526-4941-89ac-82449fdaf791 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.706978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.708007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.708640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.709119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.720557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:47.722440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:47.738173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:47.739176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:47.740046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:47.740046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:47.740046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:47.742360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 407/837] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:47.748408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.863678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:47.867343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd832-c100-40bf-8ed9-e3c5b3f0ea77 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:47.902219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:47.903112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:47.903518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:47.903916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:47.904909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:47.905358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:47.907551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b1636-f364-49a5-9268-8721f32540fd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:47.926565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 431/838] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:47.932213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:47.948691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.041137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85297-fd20-4851-a8dc-1af0b3ae99f4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.044951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fb1c-5700-4a89-a513-9a4c0f0ae66b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.045232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fb1c-5700-4a89-a513-9a4c0f0ae66b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.045501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fb1c-5700-4a89-a513-9a4c0f0ae66b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.046838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 432/839] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:48.052937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.053443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.053909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.072186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:48.074006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:48.075896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.076550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.076834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.077106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.084826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.086261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:48.097530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:48.098542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 28 10:03:48.099466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:48.103443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 408/840] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:03:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:48.104339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.156942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 433/841] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:48.180895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.215795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620344-413b-4aa6-aba5-4e161adbf725 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.249841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.250826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.251304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.251783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.252750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:48.253301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.254925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.275159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 409/842] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:48.280562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.372978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.384703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02477d-7539-456a-a036-c86c5da04207 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.423691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.424310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.424615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.424877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.430566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.494307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.495743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.531235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d585a-64a5-4deb-b848-24d1cb27ee8c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.561227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.571264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.572182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.572720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.573191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.573987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.575546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:48.585734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:48.586673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:48.587484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:48.595099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 410/843] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:48.596166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.606829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 434/844] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:03:48.607147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.612153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.708370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181760-8a74-42b7-9dfd-f7660da85d60 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.744885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1c6d5-b9cf-4fb4-8d7d-a9f339e07d6e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.745839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.746455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.746738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.747010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.747648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:48.747956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.749570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.759000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 411/845] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:48.759940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.793031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.794876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.795426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.795836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.809339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.811848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.820094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:48.832338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:48.833330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:48.834181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:48.834181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:48.834181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:48.836669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 435/846] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:48.845476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:48.875195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133783-e882-4c8c-a69b-59b23df3cb4c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:48.900024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:48.915607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:48.916578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:48.917054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:48.917496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:48.918487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:48.918855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:48.920675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:48.930966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 412/847] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:48.971695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b47b43-b84c-42ad-922b-28f290ee31fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.014188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.015186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.015684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.016174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.017066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:49.017592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.019277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.033697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 436/848] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:49.041811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.146439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13092bf-c4e5-4c93-8fb9-9bef83a04f1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.164026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:49.182560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.183192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.183473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.183748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.191002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.192448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-86c7f023-490c-42d3-bda9-01cdfa0bee01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:49.204166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:49.205093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:49.206040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 413/849] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:49.211507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.237136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2f081-dfd1-4eef-ac96-a8b2c17bc2f9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.237476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2f081-dfd1-4eef-ac96-a8b2c17bc2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.237707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2f081-dfd1-4eef-ac96-a8b2c17bc2f9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.239323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 414/850] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:49.244254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc414-b77f-486b-b90c-be1db2882225 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.244563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc414-b77f-486b-b90c-be1db2882225 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.244838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc414-b77f-486b-b90c-be1db2882225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.262867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e35bc414-b77f-486b-b90c-be1db2882225 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:49.264049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e35bc414-b77f-486b-b90c-be1db2882225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:49.315969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:49.318913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6da36e-e687-4c4f-b44b-1d5e7ab291a6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.338736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 415/851] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:49.340037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ecaf-a3d7-4782-acf2-048f679d631f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.340296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ecaf-a3d7-4782-acf2-048f679d631f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.340590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ecaf-a3d7-4782-acf2-048f679d631f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.346257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 416/852] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:49.348375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.348676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.348945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.360319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.361278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.361794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.362252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.363197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:49.363739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.366221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.370183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:49.371362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:49.385137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 437/853] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:49.387672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.468532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 417/854] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:49.503638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73268eec-4934-4e69-8330-06a55d6fa835 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.537074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.537726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.538000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.538310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.539038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=7c9ccc207322482e818b7c9f49aeed29) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:49.539335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.540788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.547283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 438/855] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/role_assignments?user.id=7c9ccc207322482e818b7c9f49aeed29&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:49.612707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.656250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:49.726629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126efee-527b-4f3c-9b53-ab6a9ff19e67 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.740063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.764263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.764936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.765224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.765530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.766124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.767955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.770960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:49.777056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:49.778024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:49.779581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 418/856] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:49.784094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.862970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53587-bf9e-4b02-9ad1-e1b692ebc2ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.891457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493dfb-9c95-4e8d-8c19-a0664a31bc37 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:49.893020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:49.902165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.903412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.903687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.903950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.915282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:49.921434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:49.922355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:03:49.923565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:49.927743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 439/857] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:49.928924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:49.931128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:49.931744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:49.932020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:49.932288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:49.932965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:49.933268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:49.934643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:49.939870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 419/858] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:49.945621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.019788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:50.038158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24c61a-6743-4c24-af75-88e5be49a331 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.050991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4522-68fe-46e4-ad7e-7aed2c5a20a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.077579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.078669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.079083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.079485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.080300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:50.080865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.082498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.089423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.090087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.090456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.090632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.091272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:50.091572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.092981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.098035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 440/859] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:50.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 420/860] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:50.100900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.220576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e887-e224-4768-9ab6-995e121fa829 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.259748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.260660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.261100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.261511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.273778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.275478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.282781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:50.290981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:50.292134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:50.293376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:50.293376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:50.293376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:50.293376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 441/861] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:50.298945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.329981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:50.400130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d83d1d-f61d-4275-a9eb-056f6e97c53e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.434153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.434767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.435042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.435310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.435958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:50.436257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.437658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.450291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 421/862] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:50.455814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.564569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8253-d2b2-4e6f-b8b1-f2dfbb457496 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.601788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.602437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.602714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.602985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.610971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.612620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:50.623969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:50.624970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:50.626483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 442/863] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:03:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:50.630909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.729666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:50.739700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a535c-d98b-4ae3-9227-ee5b8aa597a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.777494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.778114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.778394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.778659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.779391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:50.779690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:50.781198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0814f7-e786-4f5f-a942-6001907be8be None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:50.799359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 422/864] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:03:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:50.805630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:50.864535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:50.897152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958874df-dfd7-460e-b0f2-24a9c4c74cc5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.897542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958874df-dfd7-460e-b0f2-24a9c4c74cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.897819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958874df-dfd7-460e-b0f2-24a9c4c74cc5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.899569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 423/865] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:50.907878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f09d-6f6d-4ece-8731-f95fc7718cc3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:50.908629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.909005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.909502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:50.935194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:50.936889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:50.946892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:50.947555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:50.947853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:50.948181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.019730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 424/866] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:51.021152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.022650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.085478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:51.127790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 443/867] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:03:51.133357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.217190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:51.245297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fae40-1fdc-4ea3-b238-58d283baaf2a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:51.280269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:51.280669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.280966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.281250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.281524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.292724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.294938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:51.316329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:51.317545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:51.318631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:51.318631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:51.318631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:51.321148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 425/868] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:51.326089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.406917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60e97-85aa-4d82-8964-4a8dcd80b2fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:51.451433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:51.451762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113b7e3-6320-479e-944a-a4b7ca25b25a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:51.452282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.452699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.453060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.454035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.456460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.479126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 444/869] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:03:51 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:03:51.498202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:51.512151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064be558-3812-401b-9916-43429979bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:51.513134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064be558-3812-401b-9916-43429979bde5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.513569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064be558-3812-401b-9916-43429979bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.513973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064be558-3812-401b-9916-43429979bde5 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.514871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064be558-3812-401b-9916-43429979bde5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:51.515363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064be558-3812-401b-9916-43429979bde5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.517010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064be558-3812-401b-9916-43429979bde5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.531282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 426/870] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:51.537233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.559426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:51.609810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9e58-c4c9-41ea-83fa-8ffe00848d52 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.610132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9e58-c4c9-41ea-83fa-8ffe00848d52 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.610358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9e58-c4c9-41ea-83fa-8ffe00848d52 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.612147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 427/871] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:51.617947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.618269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.618609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.638025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:51.639324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:51.663047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8daf6d-ccdc-406e-8e52-6adf0d51af7a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:51.709602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:51.710557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.711157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.711429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.718447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.719899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.720577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 428/872] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:51 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:51.736207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:51.737161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:51.738198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 445/873] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:51.743434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.850081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6ae9d-2a8e-433c-bdd3-941379799e8e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:51.868234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:51.887876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:51.888825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:51.889325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:51.889789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:51.890692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:51.891220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:51.892931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:51.905873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 429/874] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:51.911664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.986908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:51.995144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:52.026937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30ef2a-ff5b-4add-9c34-1e483c547fe7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:52.069964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:52.070568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.070841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.071103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.071823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=207eced18c9b455186567c94927a9754) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:52.072121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:52.073595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e6122-6365-4332-a805-4c273bfa3159 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:52.080125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 446/875] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/role_assignments?user.id=207eced18c9b455186567c94927a9754&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:52.109215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ac86-0796-4fef-b4a4-81f1766b1219 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:52.148429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:52.149111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.149386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.149661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.150323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:52.151710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:52.161646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:52.162670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:52.163482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:52.167351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 430/876] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:52.169085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce4d1-1d37-483d-bda1-65748e061e09 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.169706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce4d1-1d37-483d-bda1-65748e061e09 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.170138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cce4d1-1d37-483d-bda1-65748e061e09 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.171341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:52.171994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 447/877] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:52.178150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.178748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.179219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.198383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:52.199976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:52.220282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:52.295148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 448/878] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:52.296481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca036d-bb87-4d5d-af78-50e15dc6c841 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:52.330213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:52.330819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.331094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.331383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.332004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:52.332301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:52.333740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:52.338622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 431/879] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:03:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:52.343982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:52.414709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:52.446585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646364c-7e22-4633-993e-85792a87ccdd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:52.486252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:52.486868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.487145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.487504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.488141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:52.488440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:52.489868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec45a8-1c4f-4dfd-a6d5-25098d8e0e5e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:52.494885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 449/880] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:03:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:52.570368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:52.689282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ddf6d-7c5e-4f1b-9499-d20ee961cb8d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:52.692682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:52.730021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:52.731001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:52.731471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:52.731958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:52.732851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:52.734414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5a6ea-046d-44d3-9b60-6aa6b04b2ab1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:52.753061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 432/881] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:03:52 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:53.548338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d1e2-7792-485f-b31a-dbbd211a9018 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:53.549430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d1e2-7792-485f-b31a-dbbd211a9018 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:53.549996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5d1e2-7792-485f-b31a-dbbd211a9018 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:53.551812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 450/882] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:53.557239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:53.557737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:53.558217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:53.576803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:53.589928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:53.664240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 433/883] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:53.935823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5d1e2-7792-485f-b31a-dbbd211a9018 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.040623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5d1e2-7792-485f-b31a-dbbd211a9018 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.076211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.077338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.077826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.078316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.079139 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.081660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.086235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3ec613b4-6602-43bc-b09b-1ff235c00a0e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:54.094038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:54.095038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:54.095878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 451/884] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:54.101860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.157712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f69b90-64be-4817-b24a-2517e2637f0c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.158068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f69b90-64be-4817-b24a-2517e2637f0c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.158247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f69b90-64be-4817-b24a-2517e2637f0c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.159911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 452/885] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:54.165172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.165455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.165726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.185077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:54.186426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:54.202433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.216030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f63dc-a8df-4ac3-b6ee-aebab5574f4c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.253950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.254558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.254833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.255101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.255923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:54.256224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.257725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.263066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 434/886] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:54.268766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.279178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 453/887] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:54.379687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae5391-4159-443f-80a9-96d947634160 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.414865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.415487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.415771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.416045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.416708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:54.417013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.418392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b9cc8-90ae-4009-ab69-6c89316081d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.424241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 435/888] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:54.430292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.450392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33926-a702-4069-8015-f0ad04d15cc4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.450590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33926-a702-4069-8015-f0ad04d15cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.450881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33926-a702-4069-8015-f0ad04d15cc4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.452385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 436/889] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:54.457311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.457569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.457858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.478265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:54.479447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:54.494336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.552166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 437/890] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:54.557425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.557576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dfdd-e249-42a0-9615-ad49143f6aa2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.597339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.606458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.607616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.608095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.608612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.610660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] RBAC: Adding query filter params (interface=public, service_id=6795996e3a004dd6b3557268855bad0b, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:54.611203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.612856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.620446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 454/891] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/endpoints?service_id=6795996e3a004dd6b3557268855bad0b&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:54.627548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.683624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aee7d-4a4a-45c5-992e-f39a3a76f3ff None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.722641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.723615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.724114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.724631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.738118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:54.745114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:54.746254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:03:54.747175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:54.749610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 438/892] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:54.755370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.760521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf08e58-77e4-4caa-9ba5-b3062bf2d29a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.768819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.798267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.798875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.799146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.799411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.800060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.801476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29507711-45ab-4a2d-b252-0ac7f75d6870 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.812606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 455/893] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:03:54.812606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab01b0-482e-4005-b9a8-7427b7eee298 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.812606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab01b0-482e-4005-b9a8-7427b7eee298 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.812606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab01b0-482e-4005-b9a8-7427b7eee298 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.812606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 456/894] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:54.815866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab01b0-482e-4005-b9a8-7427b7eee298 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:54.875909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8079-f2a3-4e69-a19f-33697b2d8b77 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.883296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:54.915269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.915891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.916222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.916492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.917253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:54.917599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.918981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7650-bf06-4e29-b9da-8a918c27a301 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.929576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 439/895] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:54.934466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab01b0-482e-4005-b9a8-7427b7eee298 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:54.935795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.936226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.936364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.956478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:54.957716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:54.974987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:54.975561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:54.975839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:54.976110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:54.976822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:54.978204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:54.995709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 457/896] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:03:54 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:03:54.996974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.037184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.051822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 440/897] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:55.113089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2822-00c7-466a-bd1d-93bdfb2f0769 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.133098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.152231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.152909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.153185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.153453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.165056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.166540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:55.181824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:55.182772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:55.183632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:55.183632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:55.183632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:55.183632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 458/898] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:55.189008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.289798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51a47-dc2e-4e3b-b20d-a12c389ef4ad None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.328881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.329496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.329769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.330104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.330752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:55.331119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.332684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.332951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.349055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 441/899] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:55.351289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.409574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.440804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b361-0f4c-4a07-a322-68376a99cd73 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.455843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f1d66-7c68-4375-af90-0f6a4a501e3a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.476175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.476852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.477131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.477404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.477997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.479383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:55.488802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:55.489811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:55.491068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:55.491792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.491792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.491792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.491989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.495187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 459/900] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:55.496072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.503077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.504545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:03:55.516084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:55.518331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:03:55.520584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:55.528525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 442/901] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:55.528525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.532803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.615516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb9cd-17dc-4452-87fd-b4ca48163860 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.625487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c84d8-fa10-4851-8bc8-c29ee2fc4478 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.641672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.653019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.653856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.654282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.654721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.655677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:55.656120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.657813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.660741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.661353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.661627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.661965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.662624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:55.662922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.668619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.669077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 460/902] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:55.670755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.688174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 443/903] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:55.689528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.771909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.776455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712d669-a4d9-48d8-b622-feec7bace687 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.791149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175470-e561-47bd-9fc8-1c172f34a9bc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.813209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.814059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.814482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.814980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.815788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:55.816259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.817810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.828162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 461/904] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:55.829961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:55.833128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.833746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.834023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.834288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.879182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:55.906878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.908334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.936602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79ad84-89ec-4209-9184-15544987344a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:55.973096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869539-3597-4733-821e-6827dcefacff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:55.974152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869539-3597-4733-821e-6827dcefacff None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:55.974649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869539-3597-4733-821e-6827dcefacff None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:55.975117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869539-3597-4733-821e-6827dcefacff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:55.976015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869539-3597-4733-821e-6827dcefacff None admin] RBAC: Adding query filter params (interface=public, service_id=896dfc668e574255a365b72f87fc9670, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:55.976535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869539-3597-4733-821e-6827dcefacff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:55.978139 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869539-3597-4733-821e-6827dcefacff None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:55.988311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 462/905] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/endpoints?service_id=896dfc668e574255a365b72f87fc9670&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:55.989458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869539-3597-4733-821e-6827dcefacff None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.017196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 444/906] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:03:56.018468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.037209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:56.094784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:56.108326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869539-3597-4733-821e-6827dcefacff None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.121757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbc23-027e-453f-886c-663bfa2e6943 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.147159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.147700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.147970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.148236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.148971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.150468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.157113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 463/907] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:03:55 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:03:56.157256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.157886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.158300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.158575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.171125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.172652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:56.189296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:56.190780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:56.191649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:56.191649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:56.191649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:56.195681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 445/908] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:56.197205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.299989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:56.308939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e999900-2e1d-454b-8c94-49525592614a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.349088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.349704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.349982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.350251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.350900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:56.351200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.352621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.365056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 464/909] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:56.370745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.471872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d3777-05b6-44dc-95c5-1627c09642e6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.497302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:56.506490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.507417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.508036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.508311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.515407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.516900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:03:56.528894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:56.529903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:56.530805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 446/910] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:56.536410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.641718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2959673-ee43-418e-9556-e84c20043dd7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.677394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.678004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.678278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.678545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.679236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:56.679528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.681002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743718-cdc1-462e-8929-77f8850af593 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.698533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 465/911] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:56.701610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:56.804344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f5ed-e6fb-4f27-8ea7-d438ca760a71 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:56.838896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:56.839507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.839779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.840045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.840805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] RBAC: Adding query filter params (role.id=4ab1bcefe40a49059d9afa89181ebf0c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=207eced18c9b455186567c94927a9754) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:56.841109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:56.842540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9066cd-3f1d-4392-9c6e-3b055b8db656 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:56.852626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 447/912] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:03:56 2022] GET /identity/v3/role_assignments?user.id=207eced18c9b455186567c94927a9754&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4ab1bcefe40a49059d9afa89181ebf0c => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:56.870133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af339b-4b97-4f50-8ad4-ebe0e3a76aa8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.870767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af339b-4b97-4f50-8ad4-ebe0e3a76aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.871236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af339b-4b97-4f50-8ad4-ebe0e3a76aa8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.873263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 466/913] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:03:56.879550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.880192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.880721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.900095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:56.901727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:56.954454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:56.959374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8491675-bac1-4c28-b320-cdc0a7d35c18 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.959660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8491675-bac1-4c28-b320-cdc0a7d35c18 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.959918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8491675-bac1-4c28-b320-cdc0a7d35c18 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.961790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 467/914] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:56.967240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:56.967502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:56.967772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:56.985500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 448/915] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:56.991026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:56.992250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:56.995822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:57.068820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 468/916] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:57.257765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.332429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.370375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669450d0-1c53-4876-9a65-1175589e1e5f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.414359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.415258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.415537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.415806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.416397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.418307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:57.429818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:57.431317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 28 10:03:57.432699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:57.440532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 449/917] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:57.440532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.441700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5642c-7b33-42ab-9611-a9309965da74 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.486405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:57.491095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.492769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.493048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.493312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.493900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.495783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:57.508683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:57.510211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:03:57.511271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 469/918] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:57.516225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.521369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:57.552967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531113-bf26-4b85-a9d6-c40da3caa765 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.597704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.598544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.599093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.599570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.600484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:57.600979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.602517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.608222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 450/919] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:57.615284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.640709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdddacb-f515-454d-82c9-e2b496b04968 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.677981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.678624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.678851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.679070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.679695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:57.679993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.681432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.686752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 470/920] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:57.695041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.753842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f474-8d97-425b-bee2-37fa097d3d54 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.763640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:57.775313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:57.798288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.799612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.800160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.800670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.801581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:57.802599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.804813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.817596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b124252-ea6c-4aef-868f-06b76153ecb1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.817780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 451/921] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:57.820050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.852999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.853516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.853799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.854063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.854697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:57.855000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.856611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:57.866779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 471/922] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:03:57.869310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:57.934877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50488b-0d66-45fa-9fbd-983594a4123c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:57.972986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:57.973797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:57.974168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:57.974534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:57.975306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:57.976803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:58.008532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 452/923] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:03:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:58.008770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81168d0-c606-40db-8408-d36db1a1e1a1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:58.042447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:58.054963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:58.055654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.056393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.056697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.057475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] RBAC: Adding query filter params (interface=public, service_id=474470ba5e604714aff441c3edae85ca, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:58.057795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:58.059298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4773e-8e62-473a-beb7-868b36a33fce None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:58.066023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 472/924] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:03:57 2022] GET /identity/v3/endpoints?service_id=474470ba5e604714aff441c3edae85ca&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:58.072069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:58.110293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:58.161968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d082-fe75-4554-b4f5-14b8a45f4905 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.162545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d082-fe75-4554-b4f5-14b8a45f4905 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.162991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d082-fe75-4554-b4f5-14b8a45f4905 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.164882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 473/925] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:58.173038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.173298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.173693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.195341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:58.196575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:58.198995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48a199-7524-455a-b337-d92d22629725 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:58.239650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:58.240437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.240772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.241040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.241654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:58.243079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:58.256549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 453/926] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:03:58 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:03:58.287104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 474/927] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:03:58.554172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:58.654957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:58.664789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dab3aa-c4bc-4e8e-a1e3-5b9d5338011a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:58.699738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:58.700753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.701278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.701767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.702589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:58.704146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2c47dcc8-a124-4ad7-9f2d-e3d457774281 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:58.713555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:58.714639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 28 10:03:58.715694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:58.718138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 454/928] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Aug 28 10:03:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:58.726086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:58.769232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a293a6-20a0-4698-a232-7a8aa5149380 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.769766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a293a6-20a0-4698-a232-7a8aa5149380 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.770041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a293a6-20a0-4698-a232-7a8aa5149380 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.771634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 455/929] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:03:58.776638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.776995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.777363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.796548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:03:58.797889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:03:58.838663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a01c8-f495-427e-83bd-09e6654e704f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:58.866010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:58.876608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 456/930] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:03:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:03:58.882889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:58.883503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:58.883780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:58.884049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:58.884728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:58.885194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:58.886595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:58.891380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 475/931] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:58.897157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:58.979019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:58.994878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b280b-3f65-419d-ad26-9be618fa0fcb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.035284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.036271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.036850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.037320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.039719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:59.040251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:59.041933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:59.048041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 457/932] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:03:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:59.053490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.152647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcfb71-5f19-4767-899d-915711d1ea5c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.187468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.188111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.188374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.188482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.188701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.189370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:59.190943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:59.214384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 476/933] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:03:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:03:59.308168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2f2d1-300f-4b4e-9531-26390059be16 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.353007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.354051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.354566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.355034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.366866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:03:59.372665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:59.373612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:03:59.374423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:59.377077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 458/934] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:03:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:03:59.383397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.433321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:59.492008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b51b3-9f43-4e4b-a122-8a7f250a2dd3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.525524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.526141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.526417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.526684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.527354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:59.527647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:59.529105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:59.537744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 477/935] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:03:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:59.543530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.564852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:59.649838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e64e8-87f4-4c3b-a387-96bdb36438a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.687049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.687598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.687923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.688197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.699194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:59.700684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:03:59.716177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:03:59.717186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:03:59.718502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:03:59.718502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:03:59.718502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:03:59.718502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 459/936] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:03:59.723541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.824912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058dcd-0f37-48fb-bbbd-434af80a071e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:03:59.827258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:59.860951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:03:59.861561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:03:59.861837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:03:59.862105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:03:59.862804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:03:59.863102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:03:59.864564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9a467-cb3f-46e4-938f-23e59fdb513c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:03:59.876395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 478/937] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:03:59.881857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:03:59.942365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:03:59.985125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95094-f06b-4aac-92f4-1426eac68924 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.020264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.020988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.021269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.021591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.029261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.030984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.041830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e56ef-477d-436b-b52b-2000eee204dd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.042120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e56ef-477d-436b-b52b-2000eee204dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.042399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e56ef-477d-436b-b52b-2000eee204dd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:00.043029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:00.044727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:04:00.046259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.046812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 479/938] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:00.047151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 460/939] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:03:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.050047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.050331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.050502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.050667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.070648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:00.071848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:00.147694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 480/940] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:00.150560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83111ebe-885f-486a-a7e3-c878a71c380e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.182294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.185754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.186375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.186906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.187189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.187929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:00.188236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.189755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cab8e-94e2-40f2-83fb-906f8bca3619 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.207776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 461/941] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:00.213330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.307052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.314077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d71d8-e028-4af4-b84e-24a013f2750e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.328362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3025068-d815-4990-8f0e-402334ed2094 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.328699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3025068-d815-4990-8f0e-402334ed2094 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.328944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3025068-d815-4990-8f0e-402334ed2094 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.330446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 462/942] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:04:00.336065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.336422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.336808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.356655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.357181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:00.357785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.358263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.358668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:00.358883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.371982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.373486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:00.392792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:00.393816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:00.394740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:00.394740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:00.394740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:00.398887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 481/943] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.400321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.448837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 463/944] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:00.450765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.505755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff82da0-06f2-4333-af29-15f97e19e157 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.540741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.541448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.541729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.541996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.542704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:00.543000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.544741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.558127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 482/945] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:00.563350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.568473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9e910-5ac2-4e50-a7f5-97d86394255e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.576820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.583415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.612556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.613456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.613915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.614362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.615183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.616915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.619717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING keystone.server.flask.common [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 28 10:04:00.639585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.sql.core [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 28 10:04:00.639585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 28 10:04:00.639585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [parameters: {'id': '809f5b76c743423f8e7a95e5d2f7f2d6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 28 10:04:00.639585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81698) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 809f5b76c743423f8e7a95e5d2f7f2d6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 809f5b76c743423f8e7a95e5d2f7f2d6. Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:04:00.663631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 28 10:04:00.664536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application yield resource Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.rollback() Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.force_reraise() Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise self.value Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application session.commit() Aug 28 10:04:00.665479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._prepare_impl() Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.flush() Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._flush(objects) Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application compat.raise_( Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application flush_context.execute() Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 28 10:04:00.666367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rec.execute(self) Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 28 10:04:00.668899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:04:00.671067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [parameters: {'id': '809f5b76c743423f8e7a95e5d2f7f2d6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.673285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.675190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 809f5b76c743423f8e7a95e5d2f7f2d6. Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 464/946] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Aug 28 10:04:00 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.678033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.709375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809b6a2-156b-4023-adc9-f9c891d47acd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.746820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.747460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.747786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.748063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.755147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.756788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:00.768225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:00.769319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 483/947] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.770193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.796937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5358ab-8d09-4a1c-b15f-44c8260c30b2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.831174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.831820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.832097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.832370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.839249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.840496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.840956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:00.853327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:00.854267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 465/948] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.860913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.879384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35e257-3e13-49c9-9fbc-6b058ec3ef8e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:00.884428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:00.915743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:00.916376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:00.916682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:00.917235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:00.917530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:00.918944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:00.929133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:00.930129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:00.930962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 484/949] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:00.932484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:00.978791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ad2c-26e5-4352-8de9-c37782b914b8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.015134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.015676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.015937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.016208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.016936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:01.017244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.018722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.038718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 466/950] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:01.038718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.043749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f0eac-c305-45c3-8139-4326d46a7cde None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.074536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:01.081237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.081848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.082125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.082392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.083053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:01.083352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.084825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.112536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 485/951] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:01.112536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.136092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:01.162378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2308d9-8933-4437-b189-9b20c44ffa27 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.209215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.210372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.210719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.210971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.212054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:01.212425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.214346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.224681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 467/952] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:01.227331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.243129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ad5f7-80ec-43ff-8bfb-37da0f0d7efd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.285917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.286440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.286777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.287054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/207eced18c9b455186567c94927a9754/roles/4ab1bcefe40a49059d9afa89181ebf0c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.289216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=207eced18c9b455186567c94927a9754, role_id=4ab1bcefe40a49059d9afa89181ebf0c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.290661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf3bb7-83cf-4d49-a661-c0ac8c7e834b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.310585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 486/953] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:04:01 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/207eced18c9b455186567c94927a9754/roles/4ab1bcefe40a49059d9afa89181ebf0c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:04:01.311727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f131f3-57d2-4db5-92f0-d61e8e92afe5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.312010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f131f3-57d2-4db5-92f0-d61e8e92afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.312282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f131f3-57d2-4db5-92f0-d61e8e92afe5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.313613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 487/954] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:01.319498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.319784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.320028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.339225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:01.340470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:01.359207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374c811-f322-4be8-b4d7-d9cd30c7d882 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.403962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:01.405551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.406186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.406460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.406726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.407427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:01.407731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.409333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.416244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 468/955] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:01.422599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.439318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:01.441939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 488/956] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:01.533815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b3955-a021-426d-9560-35e48fdf202d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.568146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.568705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.569247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.569247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.569896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.571258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.600131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 469/957] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:04:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:04:01.716694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.820677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb714c3a-0298-4141-ba0f-6bbfbcb24277 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:01.859976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:01.860662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:01.860896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:01.861162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:01.861745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:01.863165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:01.872152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:01.873332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:01.874230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 489/958] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:01.879752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:01.950866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:01.988975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97658a56-9296-471e-9a37-85afce8e766b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:02.026819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:02.027582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:02.027955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:02.028336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:02.029106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:02.029507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:02.031010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23828268-1921-4a7d-a8dd-2c6d004658b6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:02.036219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 470/959] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:02.041960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:02.085240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:02.149630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd6d0-8967-4435-b571-fcd56d75d730 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:02.184658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:02.185271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:02.185545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:02.185831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:02.186459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:02.186759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:02.188198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166e95d-f273-4821-b275-1c9c95119c98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:02.193260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 490/960] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:03.181937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf73b70-a354-4e76-b72a-06f6097adfb7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.182500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf73b70-a354-4e76-b72a-06f6097adfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.182500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf73b70-a354-4e76-b72a-06f6097adfb7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.184219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 471/961] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:03.189661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8588-d4e2-4ea1-bb07-1e4b56fb90e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.190071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8588-d4e2-4ea1-bb07-1e4b56fb90e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.190257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8588-d4e2-4ea1-bb07-1e4b56fb90e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.209689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ff6a8588-d4e2-4ea1-bb07-1e4b56fb90e5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:03.210887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ff6a8588-d4e2-4ea1-bb07-1e4b56fb90e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:03.287524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 491/962] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:03.289193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af348-d2ba-456f-a27f-ae2d62b4de81 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.289491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af348-d2ba-456f-a27f-ae2d62b4de81 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.289763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af348-d2ba-456f-a27f-ae2d62b4de81 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.291372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 472/963] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:03.298170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.298457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.298731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.317234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:03.318477 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:03.345799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:03.388008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 492/964] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:03.546346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af348-d2ba-456f-a27f-ae2d62b4de81 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:03.651751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af348-d2ba-456f-a27f-ae2d62b4de81 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:03.661618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:03.686623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:03.695157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:03.695777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.696055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.696323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.697255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:03.699615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:03.718164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.sql.core [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 28 10:04:03.718164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 28 10:04:03.718164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [parameters: {'user_id': 'fa104be554944b5fa515c12ae0bfcfe2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 28 10:04:03.718164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81698) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-24fc81b2-1766-4bd7-827e-9e2e35c1d8eb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Aug 28 10:04:03.725587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:04:03.726530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application yield resource Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.rollback() Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.force_reraise() Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise self.value Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application session.commit() Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 28 10:04:03.728315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._prepare_impl() Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.flush() Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._flush(objects) Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application compat.raise_( Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application flush_context.execute() Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rec.execute(self) Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 28 10:04:03.729475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 28 10:04:03.730382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 28 10:04:03.731256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa104be554944b5fa515c12ae0bfcfe2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:03.733963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 28 10:04:03.736486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 473/965] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Aug 28 10:04:03 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39d05-cf50-4482-9e2d-78ab1ab97ca5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39d05-cf50-4482-9e2d-78ab1ab97ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.739088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39d05-cf50-4482-9e2d-78ab1ab97ca5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.739824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 474/966] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:03.739824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39d05-cf50-4482-9e2d-78ab1ab97ca5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:03.776217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae04647-d53d-4f57-becc-97ac7b1d30be None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:03.811533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:03.812085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.812363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.812666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.822838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a86505c4-5339-4506-a39b-e8034bb0457f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:03.827692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:03.828700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:03.830128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 493/967] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:04:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:03.830930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.830930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.830930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.848123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:03.849343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:03.855831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39d05-cf50-4482-9e2d-78ab1ab97ca5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:03.867315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:03.899439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:03.900359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:03.900798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:03.901259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:03.912443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:03.914965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:03.928531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 494/968] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:03.928531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:03.936415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:03.937370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:03.938302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 475/969] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:03.944990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.011863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.037117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf35bda-47be-48ad-9300-36aa87a714e7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.073983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d4d92-de9f-484c-a4c8-435cfdef2369 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.075593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.076397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.076778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.077114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.077857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.078224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.079679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.090074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 495/970] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:04:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:04.095367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.114297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.115223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.115720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.116212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.116712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.118025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.118537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.120224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0752-1e1f-4b1d-92e1-c51944661841 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.155631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 476/971] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Aug 28 10:04:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:04.156990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87abfee-b86f-4370-a044-e75392c60b39 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.157286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87abfee-b86f-4370-a044-e75392c60b39 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.157558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87abfee-b86f-4370-a044-e75392c60b39 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.159149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 477/972] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:04.171574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.171933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.172130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.198157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:04.200110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:04.211341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ceccc-1af2-4648-8c47-327ec54d29b6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.250604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.251260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.251536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.251803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.263627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.265763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.279142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 478/973] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:04.280281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:04.282416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:04.283379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:04.284317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:04.284317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:04.284317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:04.284317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 496/974] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:04.290750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.349613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.384523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.420170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafb20c-52b5-46be-92d2-c45f0589d470 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.432736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3d2a-f184-4c97-b1ff-974d0fc6c605 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.464049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.464733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.465015 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.465382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.465878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.467271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.469390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.470080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.470418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.470752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.471451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.471861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.473312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:04.476305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:04.477301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:04.478156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 479/975] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:04.483477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.485476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 497/976] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:04.490721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.593675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b905-5241-42c3-9039-d4c65ea9dda2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.594762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701019a-62f8-4921-8fa8-ed23cda952ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.597120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.620511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.628880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.629358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.629492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.629757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.630008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.630108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.630281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.630545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.630642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.630927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.632311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.637041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 480/977] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:04.637793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.637928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.639485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:04.653085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:04.658858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 498/978] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:04.659929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.741558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab84ce8-f587-45f7-bfb9-718c27c6ab1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.751918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe45f7-683a-47ac-bfde-f2a2902e4be7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.774787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.775394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.775670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.775934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.776614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.777995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.784422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.785064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.785342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.785718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.786329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.786628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.788032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.793022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 481/979] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:04:04 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:04.793437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 499/980] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:04.797375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.799420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.832329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.863015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:04.917743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb055-1469-490f-867e-d22650659c5d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.929367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a718-945e-4a73-b4a3-54676e1ae2fe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:04.960598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.961326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.961607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.961876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.962590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] RBAC: Adding query filter params (interface=admin, service_id=474470ba5e604714aff441c3edae85ca, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.962903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.964300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.968237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:04.969136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:04.969419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:04.969687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:04.970592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 500/981] 10.222.0.40 () {56 vars in 1249 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/endpoints?service_id=474470ba5e604714aff441c3edae85ca&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:04.970710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:04.970710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:04.972170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:04.975767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:04.990276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 482/982] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:04.996189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:05.079703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:05.086483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b41c-d96b-47e3-9bce-51675e80e2e2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:05.101669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a1f1-3b82-4afd-b887-e04dc770e757 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:05.102818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:05.121654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:05.122365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:05.122647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:05.122914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:05.123529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:05.124938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:05.140536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 501/983] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:04:05.147715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:05.148355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:05.148672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:05.148949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:05.226404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:05.227798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:05.315059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:05.335900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 483/984] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:04 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:05.342119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:05.458208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6267c-a701-4692-a926-9f44689657cb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:05.493710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:05.494243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:05.494514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:05.494749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:05.506177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:05.507635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:05.522183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:05.523158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:05.529054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:05.529054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:05.529054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:05.529054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 502/985] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:05.530916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:05.640413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c090e38-3b20-457a-9477-ac8c31580d7e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:05.677406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:05.678017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:05.678296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:05.678684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:05.679356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:05.679656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:05.681098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:05.693653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 484/986] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:05.699288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:05.732498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:05.807034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242ebd2-5cfa-4a74-998f-51870eab1875 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:05.841193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:05.841833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:05.842109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:05.842376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:05.849351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:05.850892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:05.862823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:05.863786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:05.870072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 503/987] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:05.872267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:05.921200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:05.985428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef0ac5-5c95-455d-a9a7-534fb1c32eb2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:06.021155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:06.021767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:06.022044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:06.022315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:06.023009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:06.023367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:06.024853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05927c-ef7f-43f4-a235-bd54f2008f69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:06.043521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 485/988] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:04:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:06.045421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:06.142995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:06.153677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99994-dd16-4e46-a77e-439397871620 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:06.189486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:06.190094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:06.190363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:06.190653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:06.191390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] RBAC: Adding query filter params (role.id=4ab1bcefe40a49059d9afa89181ebf0c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=207eced18c9b455186567c94927a9754) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:06.191692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:06.193161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a46cb-1462-4000-80eb-21bffff383cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:06.204612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 504/989] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:06 2022] GET /identity/v3/role_assignments?user.id=207eced18c9b455186567c94927a9754&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4ab1bcefe40a49059d9afa89181ebf0c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:06.763574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4df-c8c5-4a63-adbc-44e3558ab114 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:06.764115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4df-c8c5-4a63-adbc-44e3558ab114 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:06.764115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec4df-c8c5-4a63-adbc-44e3558ab114 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:06.768546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 486/990] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:06.770791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2a61-2a44-4502-b7ca-640add5a9979 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:06.771202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2a61-2a44-4502-b7ca-640add5a9979 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:06.771304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2a61-2a44-4502-b7ca-640add5a9979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:06.789715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e06c2a61-2a44-4502-b7ca-640add5a9979 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:06.790900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e06c2a61-2a44-4502-b7ca-640add5a9979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:06.868525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 505/991] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:07.038104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89621f5-91ae-4ad7-9bee-d1101a7a5ad5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.038432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89621f5-91ae-4ad7-9bee-d1101a7a5ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.038700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89621f5-91ae-4ad7-9bee-d1101a7a5ad5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.040223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 487/992] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:07.045872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.046161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.046434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.065570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:07.066756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:07.129646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89621f5-91ae-4ad7-9bee-d1101a7a5ad5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.138595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 506/993] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:07.235680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89621f5-91ae-4ad7-9bee-d1101a7a5ad5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.237532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.273064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.274025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.274521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.274959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.275702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.277313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:07.286547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:07.287510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:07.288420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.289174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 488/994] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:07.295935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.315153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.397774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a93f8-2e61-41aa-b877-9fba115470b3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.401260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.433266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.433871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.434144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.434408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.435039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:07.435334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.436741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.442300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 507/995] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:07.447629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.508481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac3bd6-3c1c-4ff8-b1ea-f12b18a421fb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.526672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.543958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.545059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.545402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.545740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.546416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.547838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a03cf9-f9f7-4e30-bd98-a3947fe90f85 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.547949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:07.557636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:07.558936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 28 10:04:07.559981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:07.565968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 489/996] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:07.566434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.599341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.599949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.600217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.600480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.601136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:07.601431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.602835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.606271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.612952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 508/997] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:07.617418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.686149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42131-9479-4c74-aa74-742ef024db1d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.720926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fecb4-78e3-4f22-9ec2-cfc93927693c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.724417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.725251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.725680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.726098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.726880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:07.727328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.729313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.734685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 490/998] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:07.740346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.757270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.757884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.758165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.758436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.759147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] RBAC: Adding query filter params (interface=public, service_id=e7d3f0d3fa0a4d859c2d870a069e9acc, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:07.759455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.760896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.770570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 509/999] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/endpoints?service_id=e7d3f0d3fa0a4d859c2d870a069e9acc&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:07.771749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.782474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.840764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:07.849629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c93de6-8b17-4539-80f5-9e38158238bb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.877366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281020-204c-4ec1-b6de-b976868715e6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:07.883256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.883925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.884205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.884479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.885155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:07.885464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.886852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.891812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 491/1000] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:07.897433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:07.915132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:07.915740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:07.916011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:07.916274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:07.916936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:07.918298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ae9a8-8a6f-4075-83da-7238f941fcdd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:07.925093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 510/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:04:07 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:04:08.008894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef580b8-932f-4b5d-9637-d93d4c94ebb9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:08.046859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:08.047488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:08.047772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:08.048041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:08.048757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:08.050187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16077a-2f07-4001-9d3b-1d347b29a00e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:08.069841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 492/1002] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:04:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:04:08.160067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c31be-255a-4e2b-89b2-6199d46e940e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:08.160346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c31be-255a-4e2b-89b2-6199d46e940e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:08.160685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c31be-255a-4e2b-89b2-6199d46e940e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:08.162216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 511/1003] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:08.167915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:08.168220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:08.168477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:08.188797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:08.190048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:08.205654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:08.272175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 493/1004] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:08 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:08.558849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c31be-255a-4e2b-89b2-6199d46e940e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:08.639856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:08.677011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c31be-255a-4e2b-89b2-6199d46e940e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:08.711984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:08.712639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:08.712918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:08.713185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:08.713773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:08.715147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:08.724050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:08.725058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:08.725909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:08.728881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 512/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Aug 28 10:04:08 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:08.731092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:08.828971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1b070-c964-4d51-947b-8efa9aec07e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:08.861919 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:08.862543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:08.862827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:08.863108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:08.863727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] RBAC: Adding query filter params (name=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:08.864030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:08.865430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad379f-9668-4050-beb5-fbe583b920ea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:08.870504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 494/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:08.876037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:08.977826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f254a-65ca-43af-b63f-2081b4fb56cd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:09.013002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:09.013617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:09.013890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:09.014152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:09.014783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:09.015080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:09.016458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d75a9d-a7d0-4240-b612-d94696bb74d8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:09.018576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:09.028785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 513/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:09.813773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3cc8a-0d6d-4e18-a3eb-1ef623148f27 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:09.814150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3cc8a-0d6d-4e18-a3eb-1ef623148f27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:09.814312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3cc8a-0d6d-4e18-a3eb-1ef623148f27 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:09.815880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 495/1008] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:09.821054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08528-a61c-44f5-a77d-1b2b91f694ab None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:09.821327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08528-a61c-44f5-a77d-1b2b91f694ab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:09.821588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08528-a61c-44f5-a77d-1b2b91f694ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:09.840103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a9b08528-a61c-44f5-a77d-1b2b91f694ab None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:09.841217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a9b08528-a61c-44f5-a77d-1b2b91f694ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:09.909895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 514/1009] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:09 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:09.969428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27685-e3f2-4e39-b872-04cb7d5c03f2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:09.969955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27685-e3f2-4e39-b872-04cb7d5c03f2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:09.969955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27685-e3f2-4e39-b872-04cb7d5c03f2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:09.971509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 496/1010] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:09.976522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:09.976841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:09.977088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:09.995799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:09.997015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:10.067770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 515/1011] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:09 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:10.176004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb27685-e3f2-4e39-b872-04cb7d5c03f2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.187807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:10.283337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb27685-e3f2-4e39-b872-04cb7d5c03f2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.317030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.317659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.317939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.318193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.318773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.320147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:10.329155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:10.330115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:10.335998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 497/1012] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:10.336830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.336914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.396534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:10.417683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:10.442050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb0b32-c074-423a-ab5c-81c91efff20d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.449966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd1308-c424-4c2a-abcd-9b5992ea7031 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.477907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.478544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.478816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.479087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.480240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.481737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.483336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 516/1013] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:10.485699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.486462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.486868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.487312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.488085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:10.488560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.489233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.490260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.495837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 498/1014] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:10.501269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.602228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819b5a5-6544-47ba-98a4-dc9bfd66d587 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.608795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1453-35b0-4573-8e5d-780293445beb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.631094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:10.640191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.640850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.641128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.641398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.642054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.643592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b044e9c-aaf2-4fad-a2c9-41076d1ff1e8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.645980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.646799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.647256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.647705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.648562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:10.649075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.650434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.653012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:10.661959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 499/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Aug 28 10:04:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:10.662812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:10.672364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 517/1016] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:04:10 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:04:10.772770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c854b-bd8e-480c-94d4-90765f308cf2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:10.809407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:10.810054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.810338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.810605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.811285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:10.812703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552ab42-78be-4a70-8934-c293e7df98f4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:10.836689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 500/1017] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:04:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:10.976989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645788-bd81-4d33-a184-483e438831ef None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.977528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645788-bd81-4d33-a184-483e438831ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.977528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645788-bd81-4d33-a184-483e438831ef None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:10.979052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 518/1018] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:10.984300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b14-9d84-4c3e-bb04-e6d7d9678899 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:10.984774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b14-9d84-4c3e-bb04-e6d7d9678899 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:10.984880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b14-9d84-4c3e-bb04-e6d7d9678899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:11.003496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-76ac0b14-9d84-4c3e-bb04-e6d7d9678899 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:11.004779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-76ac0b14-9d84-4c3e-bb04-e6d7d9678899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:11.057953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:11.082032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 501/1019] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:11.349741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645788-bd81-4d33-a184-483e438831ef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:11.450112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645788-bd81-4d33-a184-483e438831ef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:11.484480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:11.485476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:11.485959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:11.486417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:11.487335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:11.489123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5920466-fb45-400e-8aad-f38ce3637429 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:11.512039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 519/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:04:11 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:12.631889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e751e-7ae8-40e8-aee6-a56bf5dfd21e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:12.632464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e751e-7ae8-40e8-aee6-a56bf5dfd21e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:12.632464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e751e-7ae8-40e8-aee6-a56bf5dfd21e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:12.633974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 502/1021] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:12.639106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70dc35-e491-45d7-8fd5-00e2cec8e6d4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:12.639523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70dc35-e491-45d7-8fd5-00e2cec8e6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:12.639714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70dc35-e491-45d7-8fd5-00e2cec8e6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:12.658282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc70dc35-e491-45d7-8fd5-00e2cec8e6d4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:12.659628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc70dc35-e491-45d7-8fd5-00e2cec8e6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:12.698399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:12.739667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 520/1022] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:12.817343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547737b-8877-4ad2-b014-bb19b2a50c60 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:12.817594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547737b-8877-4ad2-b014-bb19b2a50c60 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:12.817826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547737b-8877-4ad2-b014-bb19b2a50c60 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:12.819376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 503/1023] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:12.824542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:12.824807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:12.825104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:12.843986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:12.845095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:12.913549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 521/1024] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:13.004968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547737b-8877-4ad2-b014-bb19b2a50c60 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.115595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547737b-8877-4ad2-b014-bb19b2a50c60 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.151357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.151991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.152266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.152577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.153708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.155155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.157251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 504/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:13.162878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.174823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.228129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.261816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.268279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283ad2b-7261-4200-93ee-b59fd7eb8513 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.285023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3524-2e8a-4577-9dcd-219779aec2fc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.306021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.306636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.306911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.307176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.307841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.309349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.325342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.325973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.326248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.326584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.340527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 522/1026] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:04:13 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:13.340760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7d31ac50-497f-429a-ade6-f52fd05b6643 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:13.342738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:13.343764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:13.344635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 505/1027] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:13.350351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.424002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727877-46c7-44e9-ab94-06de403d824c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.424711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727877-46c7-44e9-ab94-06de403d824c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.425401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac727877-46c7-44e9-ab94-06de403d824c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.427888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 506/1028] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:04:13.433816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.434414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.434902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.460763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:13.463275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:13.465062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842b84a-ddf4-43f6-be30-1734a5332d44 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.506045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.506673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.506955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.507222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.507904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:13.508202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.509633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.518645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 523/1029] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:13.524440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.543878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.545302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 507/1030] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:13.637887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6c314-e0ff-4c35-8801-ccc9abec3cb9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.673314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.673943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.674282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.674502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.685383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.686841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:13.701472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:13.702439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:13.703638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:13.703638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:13.703638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:13.703638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 524/1031] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:13.708569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.751227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.812769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322bdb4-8c48-430d-ab01-523400db3663 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.814573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.849383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.850134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.850416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.850699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.851341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:13.851640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.853069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.870548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 508/1032] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:13.871292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.886747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.923326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc44917-e729-4ae5-9e4b-96c85bf56fe6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:13.958275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:13.958937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:13.959216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:13.959500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:13.960068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:13.961520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:13.971045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:13.972142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:13.973020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 525/1033] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:13.978476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:13.983202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:13.990312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509333a-4d33-460a-94d5-a0707c48f23d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.027727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.028553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.028938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.029214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.037645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.046189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:14.059248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:14.066119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:14.067367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 509/1034] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:14.068111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.100250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b5f5-c2f9-46a5-9c1c-e907ad52714a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.128468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.134823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.135426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.135706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.135985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.136636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:14.136940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.138325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.143487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 526/1035] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:14.148990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.174291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467adb07-fafc-44eb-ae43-b11b73155b23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.208025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.209032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.209557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.209968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.210826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:14.211360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.213068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.235443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 510/1036] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:14.236556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.247179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801446b-f0c5-451e-86e1-4600930aa789 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.256013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.283659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.284406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.284714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.284981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.285604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:14.285900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.287274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.292199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 527/1037] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:14.298204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.339756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ed1d0-d20d-4258-b03b-9307d996da80 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.362520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.377105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.377809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.378157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.378488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.410907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456e62-1e94-44dc-b61a-6036e7ba59ff None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.448127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.448764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.449044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.449313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.450025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] RBAC: Adding query filter params (interface=public, service_id=839239abdc4f4f95bb13d41742a3d5ca, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:14.450328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.451710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.453658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.455868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.461980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 528/1038] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/endpoints?service_id=839239abdc4f4f95bb13d41742a3d5ca&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:14.462726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.483415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.565626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6df9f-9556-45ed-8570-00c4107b4e6b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.573236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 511/1039] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:14.581089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.601930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.614340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.614954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.615227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.615493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.616105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.617529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.632576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 529/1040] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:04:14.694610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd694-6df0-47aa-bd00-b7554e185802 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.714521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:14.735632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.736288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.736602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.736874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.748434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.750024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:14.766737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:14.767812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:14.768737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:14.768737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:14.768737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:14.768737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 512/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:14.775377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:14.878216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d3216-8250-4cd5-b959-40f29fbf75a8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:14.913492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:14.914105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:14.914385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:14.914653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:14.915305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:14.915604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:14.917031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:14.929328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 530/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:14.934824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:15.039956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388297-03d8-4689-ab71-5ae231f60de6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:15.074738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:15.075635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.076072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.076486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.083489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:15.085131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:15.096408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:15.097373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:15.098242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 513/1043] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:15.103389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:15.152014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:15.213530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9b74-ae7f-42d8-aeb1-5ad271896a3e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:15.251535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:15.252157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.252435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.252737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.253437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:15.253739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:15.255193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aec9db-2919-44e0-9c20-0072f092b666 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:15.273275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 531/1044] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:15.275010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:15.291857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df046a96-9f61-4c57-819b-bfd9bd747caf None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df046a96-9f61-4c57-819b-bfd9bd747caf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.292733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df046a96-9f61-4c57-819b-bfd9bd747caf None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.294212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 532/1045] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:15.299820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.300329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.300818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.309276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:15.319703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:15.320873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:15.379418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67370ba8-3cb5-418c-bcf9-e90924543f1f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:15.396569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 533/1046] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:15.412662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:15.413273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.413554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.413821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.414551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] RBAC: Adding query filter params (role.id=4ab1bcefe40a49059d9afa89181ebf0c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:15.414860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:15.416303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:15.428805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 514/1047] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:15 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4ab1bcefe40a49059d9afa89181ebf0c => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:15.657510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:15.695231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:15.764415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751e028-63fe-4526-8e8e-e9ed83a10df3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:15.798933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:15.799598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.799881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.800133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.810345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:15.815178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:15.816077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:15.821382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 534/1048] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:15.823172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:15.920694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:15.928955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73485f-560d-4086-a317-2ea540d4527d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:15.963242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:15.963857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:15.964141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:15.964414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:15.965111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:15.965412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:15.966794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:15.975927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 515/1049] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:15.981730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:16.081521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:16.088202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776a396-20b3-4cf9-879d-44b26012cbcd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:16.123049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:16.123681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.123958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.124229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] PATH_INFO: `/v3/users/2a2185f6fa3949db99821957787fdec9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.137868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2185f6fa3949db99821957787fdec9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:16.139334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:16.141213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 535/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:15 2022] GET /identity/v3/users/2a2185f6fa3949db99821957787fdec9 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:16.147081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:16.249275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d109a-38e4-4835-882a-90fec25e64e1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:16.282653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:16.283252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.283525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.283785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.308627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:16.352395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:16.353870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ae10-4251-457a-a4ff-c46f8f84b69a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:16.457261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 516/1051] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:16.458799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:16.547858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4547190-871f-434e-a776-a92b24678351 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.548051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4547190-871f-434e-a776-a92b24678351 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.548314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4547190-871f-434e-a776-a92b24678351 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.552540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 517/1052] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:04:16.554882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.555163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.555581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.568110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22975d-8b98-4589-8222-39ec79a853f9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:16.573640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:16.574808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:16.611138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:16.611801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.612050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.612369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.613502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:16.614945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:16.616603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 536/1053] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:16 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:16.622559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:16.632450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:16.649040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:16.652562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 518/1054] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:16.729552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0e1d5-affe-462f-a265-cea62b646aca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:16.765751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:16.766353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:16.766630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:16.766894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:16.767617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=1e6d2a70d927446cbdc3f60b3359821a, user.id=2a2185f6fa3949db99821957787fdec9) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:16.767922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:16.769430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:16.780530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 537/1055] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:16 2022] GET /identity/v3/role_assignments?user.id=2a2185f6fa3949db99821957787fdec9&scope.project.id=1e6d2a70d927446cbdc3f60b3359821a&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:16.912984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.011687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9703-0587-4aca-950c-d4b409ed27c7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.046269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.046895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.047170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.047434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.048012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:17.049405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:17.058147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:17.059175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 28 10:04:17.060004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:17.064269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 519/1056] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Aug 28 10:04:16 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:17.065770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.150281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:17.166404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163308-bd62-4dd3-8ef5-f228e5b11f18 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.202780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.203398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.203674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.203942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.204607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:17.204909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:17.206366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc728b-de2f-4826-9234-5e8e5d0e3865 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:17.212225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 538/1057] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:17.217314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.271761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:17.273848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261f201-9f60-4525-a040-32d43ad9c14a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.274099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261f201-9f60-4525-a040-32d43ad9c14a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.274396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261f201-9f60-4525-a040-32d43ad9c14a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.276035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 539/1058] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:04:17.280722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.280994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.281265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.299494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:17.300752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:17.328002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bb882-b01f-42b5-9fb7-638b4e8a88fb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.362632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.363240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.363520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.363794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.364429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:17.364762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:17.366143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:17.370610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 540/1059] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:17.372540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 520/1060] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:17.376643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.457904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:17.478266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73f4a0-5924-4a82-96be-9257cb5d15a9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.512171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.512812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.513080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.513344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.514026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:17.515398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:17.537343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 541/1061] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:04:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:17.643838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.752244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce222d8-8bb1-4563-b06f-e140583b0032 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.766907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:17.793741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.794379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.795000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.795266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.805662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:17.810926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:17.811886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:17.813168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 521/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:04:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:17.818479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:17.924590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24917086-95fd-4790-bd38-6d092c476d39 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:17.957878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:17.958479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:17.958755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:17.959018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:17.959674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:17.959967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:17.961359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:17.963548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:17.972444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 542/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:04:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:17.978159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.080021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923c49e-c813-445d-9138-e85bfd7af592 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.113157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68548-a479-40a2-9544-157520995c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.113780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68548-a479-40a2-9544-157520995c15 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.114056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68548-a479-40a2-9544-157520995c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.114318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68548-a479-40a2-9544-157520995c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.124685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68548-a479-40a2-9544-157520995c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.126118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68548-a479-40a2-9544-157520995c15 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-09f68548-a479-40a2-9544-157520995c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:18.139874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:18.141031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:18.141949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:18.141949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:18.141949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:18.144847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 522/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:18.146904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.246852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be0f1-6187-4b5e-87e2-2d4e91f8bc01 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.280554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.281158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.281432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.281692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.282323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:18.282619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.284011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.296187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 543/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:18.301996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68548-a479-40a2-9544-157520995c15 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.327480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:18.403870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68548-a479-40a2-9544-157520995c15 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.438039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.438662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.438935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.439196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.446831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.448263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:18.459401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:18.460381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:18.461344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:18.464620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 523/1066] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:18.466863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.530834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:18.577523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379c064-0e8b-4f90-b50d-26227f6e52c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.615640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.616252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.616547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.616817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.617540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:18.617838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.619270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0b00-ddf2-4bde-83ae-0797324e4cdf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.639051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 544/1067] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:18.644797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.651485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13d1a-96c3-465b-9a2a-372c2a4be634 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.651762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13d1a-96c3-465b-9a2a-372c2a4be634 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.652033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca13d1a-96c3-465b-9a2a-372c2a4be634 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.653547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 545/1068] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:04:18.658271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.658474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.658769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.677035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:18.678199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:18.726772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:18.744260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 546/1069] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:18.745102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d52c5-dc5c-47b1-bbdf-660f45b4b567 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.778792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.779410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.779685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.779948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.790447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.791882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:18.806282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:18.807199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:18.808068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:18.808068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:18.808068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:18.812071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 524/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:18.813320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:18.859581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:18.913067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f216-9a98-4d1b-992f-14495dd37e21 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:18.949109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:18.949715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:18.949987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:18.950249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:18.950892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:18.951184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:18.952574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:18.964112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 547/1071] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:18.969825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.015545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.073796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8d8ac-745e-4d69-9a00-236f133cf83f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.107216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.108269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.108575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.108859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.110950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.116565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.118413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.128933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78e650-da05-4334-9f8c-2d879230be61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:19.129699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:19.130541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:19.131462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:19.135701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 525/1072] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:19.136567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.138626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.168941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.169945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.170178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.170341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.184023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:19.189027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:19.190088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:19.190999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 548/1073] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:19.195760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.245968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523304-322d-4650-b897-6e173ef155a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.281509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.282121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.282396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.282664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.283430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:19.283647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.285179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.298036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 526/1074] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:19.304256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.305546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93f1ab-93b2-4bdd-9856-622e89ea2252 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.340997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.341711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.341982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.342245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.342901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:19.343196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.344616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.353456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 549/1075] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:19.359054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.366872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.387992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.416049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b1e85-07b1-40bc-8865-7c9df2b0c81d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.452408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.453109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.453382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.453650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/4ab1bcefe40a49059d9afa89181ebf0c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.456079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=cbe80d5de4bb4746a1756d1077e647a8, role_id=4ab1bcefe40a49059d9afa89181ebf0c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.457573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779e45-6d6f-4556-8428-4b9b82e13ba9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.462229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91e6fc-fed5-474c-a506-c28c25f6be52 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.472441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 527/1076] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:04:19 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/cbe80d5de4bb4746a1756d1077e647a8/roles/4ab1bcefe40a49059d9afa89181ebf0c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:04:19.478589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423eb5c-a458-4c54-b30d-b9f1e9d320ad None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.478949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423eb5c-a458-4c54-b30d-b9f1e9d320ad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.479303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423eb5c-a458-4c54-b30d-b9f1e9d320ad None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.481255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 528/1077] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:19.487165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.487533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.487736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.501013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.501677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.501955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.502227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] PATH_INFO: `/v3/users/2a2185f6fa3949db99821957787fdec9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.503636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2185f6fa3949db99821957787fdec9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.505211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.506793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:19.506991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 550/1078] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/users/2a2185f6fa3949db99821957787fdec9 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:19.508423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:19.513590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.601110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 529/1079] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:19.623762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.631204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df10a0-fbb2-4d69-9676-b3bda07f98b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.667482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.668120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.668401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.668706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.670174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.671626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.673617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 551/1080] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:19.679682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.726698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:19.785373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32dc46-2617-4de1-81ee-f6c3854348c1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:19.819567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:19.820225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:19.820524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:19.820871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a/users/2a2185f6fa3949db99821957787fdec9/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:19.822793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e6d2a70d927446cbdc3f60b3359821a, user_id=2a2185f6fa3949db99821957787fdec9, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:19.824241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:19.839532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 530/1081] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:04:19 2022] PUT /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a/users/2a2185f6fa3949db99821957787fdec9/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:04:19.866417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:19.976892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316797f-2462-4f36-8f26-0c544da8bbfe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:20.013324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:20.013944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:20.014216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:20.014485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:20.046765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:20.085506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:20.086965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:20.187430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 552/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:20.193580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:20.291986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c31a3-65c3-4c91-ac0d-a26ffa9d92a4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:20.295927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:20.329452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:20.330386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:20.331193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:20.331646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:20.342536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:20.344274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fec0f9b3-6c5c-4ae0-bc0e-f0adcf8c71e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:20.359795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:20.360789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:20.361786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:20.361786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:20.361786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:20.362044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 531/1083] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:20.368368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:20.432651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:20.473742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f1f5-7d45-447e-885e-6f72f178b002 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:20.509853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:20.510469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:20.510720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:20.511015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:20.511661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:20.511959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:20.513373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55429c4-a060-49f4-bc6c-03610bd645fd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:20.525477 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 553/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:21.387266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c018e2-8c8b-4b58-ac39-add631a94033 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:21.387855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c018e2-8c8b-4b58-ac39-add631a94033 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:21.387855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c018e2-8c8b-4b58-ac39-add631a94033 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:21.389615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 532/1085] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:21.394731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9c9f-5908-4058-aafb-0be080129cd6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:21.395137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9c9f-5908-4058-aafb-0be080129cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:21.395272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9c9f-5908-4058-aafb-0be080129cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:21.414097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f64d9c9f-5908-4058-aafb-0be080129cd6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:21.415282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f64d9c9f-5908-4058-aafb-0be080129cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:21.483797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 554/1086] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:21.728021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423538-a773-4316-b08b-773f97840973 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:21.728224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423538-a773-4316-b08b-773f97840973 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:21.728456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423538-a773-4316-b08b-773f97840973 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:21.729845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 533/1087] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:21.735244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:21.735595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:21.735865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:21.748547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa423538-a773-4316-b08b-773f97840973 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:21.754950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:21.756170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:21.806665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:21.823479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 555/1088] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:21.851604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa423538-a773-4316-b08b-773f97840973 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:21.884905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:21.885529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:21.885808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:21.886080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:21.896842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:21.901626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:21.902587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:21.903447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 534/1089] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Aug 28 10:04:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:21.909073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.011418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f387423-190c-4f27-8ea5-8f180cc73eb1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.045258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.045854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.046133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.046399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.047060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:22.047356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.048861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.056763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 556/1090] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:04:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:22.062657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.081461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.087407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.134285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.162640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7c68-27b2-435c-83e7-6d1d50a53aa1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.188521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575bf90-d172-405c-b2a8-79d81ab30ef7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.195136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.195768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.196046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.196316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.206325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.207776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.220711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.221336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.221658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.221923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:22.222910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:22.223859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:22.224850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:22.224850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:22.224850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:22.224850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 535/1091] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:22.229996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.231928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:22.237212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:22.238420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:22.239821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:22.243155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 557/1092] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:22.243952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.296337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.329761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462245f-fd3b-4fef-8349-91ba58ac1c1e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.345121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb7cfe-e155-4624-b495-5f66c3535074 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.352093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.362331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.362991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.363270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.363535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.364175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:22.364471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.365869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.378124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 536/1093] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:22.379089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.379760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.380040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.380304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.380997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:22.381294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.382747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.383778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.391566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 558/1094] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:22.397356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.496336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d4a-b609-40af-b381-4b1939e42475 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.500578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a9e85-73ff-4cee-8fee-f7bb3cbc4013 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.527343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.533181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.533822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.534208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.534480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] PATH_INFO: `/v3/users/2a2185f6fa3949db99821957787fdec9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.535603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] RBAC: Authorizing `identity:get_user(user_id=2a2185f6fa3949db99821957787fdec9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.536368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.537031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.537128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.537311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.537582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.538730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 559/1095] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/users/2a2185f6fa3949db99821957787fdec9 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:22.544987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.547654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.549083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:04:22.560831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:22.561991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:22.563007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 537/1096] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:22.567702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.582996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.648980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a366-757a-4412-969e-b0762518d7bf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.666491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9581be9-05d2-4666-9c86-89b7df0f822c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.683446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.684055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.684330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.684622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.699516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.700105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.700386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.700683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.701416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:22.701720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.703148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.719934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 538/1097] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:22.725816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.755000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.756568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.776581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.790758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.830756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869a11-5762-470f-b0b7-1797fa672521 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.855438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 560/1098] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 28 10:04:22.861453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:22.864009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.864652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.864936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.865203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.932589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.934037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.958956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a438bd-3f48-4e36-bc7b-d22c8e940a8b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:22.978016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:22.992307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:22.992962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:22.993238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:22.993512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:22.994671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:22.996111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:22.997787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 561/1099] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:23.003976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.007384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:23.036370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 539/1100] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:23.037645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.101542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d417f9-af24-4056-b080-86ac1470bf37 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.133673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d5bd0-c70f-4b3c-81a6-a3e5320bb1a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.136434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.137072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.137343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.137601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.138310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=1e6d2a70d927446cbdc3f60b3359821a, user.id=2a2185f6fa3949db99821957787fdec9) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:23.138610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.140028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.145951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 562/1101] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/role_assignments?user.id=2a2185f6fa3949db99821957787fdec9&scope.project.id=1e6d2a70d927446cbdc3f60b3359821a&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:23.168758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.169406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.169991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.170284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.181792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.183280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.191140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:23.200957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:23.202019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:04:23.202925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:23.202925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:04:23.202925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:23.202925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 540/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:04:23.208222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.287055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:23.322408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff5a0-ed19-4083-a553-f9f6f7ae5d76 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.357659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.358170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.358444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.358709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.359349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:23.359647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.361056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.372563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 563/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:23.378513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.478355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfd85-3cce-4765-861f-11e3fde13634 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.511130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.511751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.512020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.512285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.519123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.520597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:04:23.532002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:23.532964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:23.534085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 541/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:23.539178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.644153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8027e92-4ec0-44a8-92f3-b3f8c548cdfd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.671384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:23.681685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.682305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.682583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.682854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.683552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:23.683855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.685343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0cfe0-3f41-45c2-9567-3698743b5e3c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.697632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 564/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:23.704099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:23.762944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:23.806206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf246c-e0b4-4883-ae88-af418f876296 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:23.843233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:23.843828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:23.844110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:23.844376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:23.845129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] RBAC: Adding query filter params (role.id=4ab1bcefe40a49059d9afa89181ebf0c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=cbe80d5de4bb4746a1756d1077e647a8) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:23.845433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:23.846951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f32e1-5818-491f-b0f2-72ecb973ae6a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:23.852866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 542/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:23 2022] GET /identity/v3/role_assignments?user.id=cbe80d5de4bb4746a1756d1077e647a8&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4ab1bcefe40a49059d9afa89181ebf0c => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:25.062009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c956-3a3b-4d6d-8a55-3d1f1e1c7a9d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:25.062880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c956-3a3b-4d6d-8a55-3d1f1e1c7a9d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:25.062880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91c956-3a3b-4d6d-8a55-3d1f1e1c7a9d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:25.065063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 565/1107] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:25.073275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:25.073708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:25.073797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:25.093584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:25.094773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:25.164999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 543/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:25.435541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91c956-3a3b-4d6d-8a55-3d1f1e1c7a9d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:25.551179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91c956-3a3b-4d6d-8a55-3d1f1e1c7a9d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:25.586323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:25.586795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:25.587068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:25.587336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:25.588599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:25.590047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c01c00-466a-409f-a0ea-4b8880d83474 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:25.591626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 566/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:25.598208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:25.656974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:25.705605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead3cd1-5aa0-47d1-a9e8-ffe57bac2a74 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:25.741092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:25.741689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:25.741961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:25.742232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:25.742877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:25.744315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753933a-33b8-47b6-9642-570678bb931e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:25.768689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 544/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:04:25 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:27.685240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91f6a-6553-41cf-b544-07846d7bf2fc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:27.685758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91f6a-6553-41cf-b544-07846d7bf2fc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:27.685758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91f6a-6553-41cf-b544-07846d7bf2fc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:27.687146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 567/1111] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:27.693576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:27.694267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:27.694411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:27.724530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:27.725722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:27.794332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 545/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:28.063978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91f6a-6553-41cf-b544-07846d7bf2fc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:28.132797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:28.182535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91f6a-6553-41cf-b544-07846d7bf2fc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:28.217863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:28.218475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:28.218751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:28.219016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] PATH_INFO: `/v3/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:28.220539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] RBAC: Authorizing `identity:get_role(role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:28.222112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:28.223417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 568/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:28 2022] GET /identity/v3/roles/07556e60b8b243f0844a294e71af2a62 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:28.229589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:28.271593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:28.332300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fa9a6-2584-4ea4-9b10-d8f2161dcb1d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:28.366915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:28.367530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:28.367800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:28.368061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] PATH_INFO: `/v3/users/8dee5af962a54d59ae5f356dd286e53d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:28.381472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] RBAC: Authorizing `identity:get_user(user_id=8dee5af962a54d59ae5f356dd286e53d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:28.382900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:28.384480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 546/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:28 2022] GET /identity/v3/users/8dee5af962a54d59ae5f356dd286e53d => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:28.393664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:28.518674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf49d63-930c-4944-988c-8fda142df383 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:28.542381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:28.555552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:28.556155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:28.556434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:28.556734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:28.642346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:28.643892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:28.744071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 569/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:28.753739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:28.860539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:28.871093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce80a3-2457-4e44-896b-f11090581c89 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:28.916489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:28.917131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:28.917404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:28.917665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:28.918881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:28.920783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d23f-ee92-48d9-935c-e9a2ee7602d8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:28.922385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 547/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:28 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:28.929501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:28.959902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:29.039871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4707191-6e91-4ce5-8f63-b7e9f9fe099a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:29.076951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:29.077541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:29.077818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:29.078090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:29.078813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=1e6d2a70d927446cbdc3f60b3359821a, user.id=8dee5af962a54d59ae5f356dd286e53d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:29.079124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:29.080595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b2e14-f39e-48ea-8467-ebaf5635f843 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:29.086154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 570/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:28 2022] GET /identity/v3/role_assignments?user.id=8dee5af962a54d59ae5f356dd286e53d&scope.project.id=1e6d2a70d927446cbdc3f60b3359821a&role.id=07556e60b8b243f0844a294e71af2a62 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:30.929612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce503f4-722c-43fe-b723-0cb4110b9a65 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:30.930196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce503f4-722c-43fe-b723-0cb4110b9a65 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:30.930196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce503f4-722c-43fe-b723-0cb4110b9a65 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:30.931493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 548/1118] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:30.937778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:30.938470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:30.938639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:30.963790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:30.965030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:31.037126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 571/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:31.304226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce503f4-722c-43fe-b723-0cb4110b9a65 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:31.417518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce503f4-722c-43fe-b723-0cb4110b9a65 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:31.460145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:31.460804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:31.461089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:31.461356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] PATH_INFO: `/v3/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:31.462886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] RBAC: Authorizing `identity:get_role(role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:31.464269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:31.465633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 549/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:31 2022] GET /identity/v3/roles/07556e60b8b243f0844a294e71af2a62 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:31.474091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:31.493930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:31.578079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2582b-1617-461d-ac8d-3f5be264775a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:31.619125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:31.619758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:31.620035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:31.620304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] PATH_INFO: `/v3/users/8dee5af962a54d59ae5f356dd286e53d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:31.621536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] RBAC: Authorizing `identity:get_user(user_id=8dee5af962a54d59ae5f356dd286e53d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:31.622997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:31.624621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 572/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:31 2022] GET /identity/v3/users/8dee5af962a54d59ae5f356dd286e53d => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:31.630854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:31.664600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:31.736532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde269d6-fd91-4885-8937-8d6fe4b2bd45 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:31.776143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:31.776795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:31.777076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:31.777344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:31.778577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:31.780028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7694b0-2ab2-42fc-a9b0-0cc2cad74dde None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:31.781675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 550/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:31 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:31.787821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:31.881472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:31.897955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a57b0-823c-4fa8-82d0-7434cc7d3a30 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:31.935878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:31.936549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:31.936821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:31.937088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a/users/8dee5af962a54d59ae5f356dd286e53d/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:31.939598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e6d2a70d927446cbdc3f60b3359821a, user_id=8dee5af962a54d59ae5f356dd286e53d, role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:31.941167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ef1ae-d209-4336-bae0-dcabd55ba163 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:31.956794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 573/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:04:31 2022] PUT /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a/users/8dee5af962a54d59ae5f356dd286e53d/roles/07556e60b8b243f0844a294e71af2a62 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:04:33.803539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6653902-7968-4e1e-8ae5-d7315b394808 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:33.804536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6653902-7968-4e1e-8ae5-d7315b394808 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:33.804536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6653902-7968-4e1e-8ae5-d7315b394808 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:33.806610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 551/1124] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:33.812907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:33.813650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:33.813819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:33.838498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:33.839635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:33.907456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 574/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:34.184278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6653902-7968-4e1e-8ae5-d7315b394808 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:34.287310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:34.291797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6653902-7968-4e1e-8ae5-d7315b394808 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:34.329752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:34.330368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:34.330641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:34.330909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] PATH_INFO: `/v3/roles/07556e60b8b243f0844a294e71af2a62` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:34.332367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] RBAC: Authorizing `identity:get_role(role_id=07556e60b8b243f0844a294e71af2a62)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:34.333779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:34.335092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 552/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:34 2022] GET /identity/v3/roles/07556e60b8b243f0844a294e71af2a62 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:34.341117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:34.423465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:34.450512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42633c-f487-43b1-a5c7-d44316496523 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:34.485263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:34.485881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:34.486156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:34.486420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] PATH_INFO: `/v3/users/8dee5af962a54d59ae5f356dd286e53d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:34.487686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] RBAC: Authorizing `identity:get_user(user_id=8dee5af962a54d59ae5f356dd286e53d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:34.489165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:34.490747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 575/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:34 2022] GET /identity/v3/users/8dee5af962a54d59ae5f356dd286e53d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:34.496733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:34.607884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f989e38-bac5-4470-a3f8-c4aa40ee9305 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:34.645432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:34.646034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:34.646311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:34.646580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:34.682069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:34.726327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:34.727726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:34.829830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 553/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:34.839457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:34.959899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa761b-f7b5-4bb2-acbb-777d2bb646f6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:34.995194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:34.995741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:34.996016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:34.996278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] PATH_INFO: `/v3/projects/1e6d2a70d927446cbdc3f60b3359821a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:34.997541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] RBAC: Authorizing `identity:get_project(project_id=1e6d2a70d927446cbdc3f60b3359821a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:34.998986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a464e-dc3f-476c-b1ea-e67de9cdc226 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:35.000594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 576/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:34 2022] GET /identity/v3/projects/1e6d2a70d927446cbdc3f60b3359821a => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:35.007250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:35.076063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:35.111538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50fe1-1911-4f76-b33c-05d2d17c8a50 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:35.147557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:35.148145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:35.148420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:35.148727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:35.149448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] RBAC: Adding query filter params (role.id=07556e60b8b243f0844a294e71af2a62, scope.project.id=1e6d2a70d927446cbdc3f60b3359821a, user.id=8dee5af962a54d59ae5f356dd286e53d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:35.149749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:35.151177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f821a-ad1b-43d2-abaa-e10f264e8782 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:35.157774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 554/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:35 2022] GET /identity/v3/role_assignments?user.id=8dee5af962a54d59ae5f356dd286e53d&scope.project.id=1e6d2a70d927446cbdc3f60b3359821a&role.id=07556e60b8b243f0844a294e71af2a62 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:37.053271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853f68-7618-43bf-a35b-b3e5f8e1cb47 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:37.053858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853f68-7618-43bf-a35b-b3e5f8e1cb47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:37.053858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853f68-7618-43bf-a35b-b3e5f8e1cb47 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:37.055153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 577/1131] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:37.061536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:37.062302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:37.062415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:37.088625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:37.089820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:37.159460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 555/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:37.426978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853f68-7618-43bf-a35b-b3e5f8e1cb47 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:37.444370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:37.545219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853f68-7618-43bf-a35b-b3e5f8e1cb47 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:37.581823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:37.582470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:37.582712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:37.583015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:37.584295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:37.585893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bb298-eb1a-4fd6-9768-bc4c653e49fc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:37.588149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 578/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:37.597177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:37.658989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:37.699011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2857e7e-49ec-443c-86ba-092bc10778c6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:37.734126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:37.734724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:37.734998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:37.735268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:37.735917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:37.737407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc06cf-e81b-4822-b830-185fa7918ce4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:37.767743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 556/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:04:37 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:04:39.670398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45081-bb2a-4347-ac10-5ad5c42000ab None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:39.670955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45081-bb2a-4347-ac10-5ad5c42000ab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:39.670955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45081-bb2a-4347-ac10-5ad5c42000ab None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:39.673078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 579/1135] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:39.680630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:39.681049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:39.681231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:39.705543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:39.707055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:39.781779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 557/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:40.062324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45081-bb2a-4347-ac10-5ad5c42000ab None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:40.150514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:40.175394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45081-bb2a-4347-ac10-5ad5c42000ab None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:40.208742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:40.209354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:40.209629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:40.209895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:40.211124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:40.212568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c32e82-7d92-4570-a032-47c56097e1b0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:40.214133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 580/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:40.220025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:40.267145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:40.322978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cfa3a-7397-451f-90dd-93bdf48f23c9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:40.356345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:40.356971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:40.357248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:40.357515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:40.358157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:40.359587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650f3dc-5f1b-467b-a4ce-e3c513cd6781 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:40.383037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 558/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:04:40 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:42.296233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c765e-5931-423c-bdd1-009e35c95204 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:42.297295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c765e-5931-423c-bdd1-009e35c95204 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:42.297295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c765e-5931-423c-bdd1-009e35c95204 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:42.299295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 581/1139] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:42.307298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:42.307735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:42.307825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:42.326553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:42.327722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:42.395083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 559/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:04:42.664842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c765e-5931-423c-bdd1-009e35c95204 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:42.775464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c765e-5931-423c-bdd1-009e35c95204 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:42.811516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:42.812130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:42.812404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:42.812706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:42.823686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:42.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:42.829641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:42.830870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 582/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:42.836409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:42.871168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:42.943747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc12f-a53c-4b3c-b730-0f9905bd6b9a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:42.978487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:42.979082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:42.979358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:42.979628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:42.980297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:42.980620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:42.982010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:42.990087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 560/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:42.999589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:43.012888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:43.107615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b6d90-62ca-4ccf-98e9-97ba3dbeac03 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:43.146708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:43.147320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:43.147592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:43.147858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] PATH_INFO: `/v3/users/930e053bc85341188753cb9f16f5c90b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:43.162516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] RBAC: Authorizing `identity:get_user(user_id=930e053bc85341188753cb9f16f5c90b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:43.163969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:43.165592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 583/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:42 2022] GET /identity/v3/users/930e053bc85341188753cb9f16f5c90b => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:43.172373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:43.267926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:43.278491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6f2b6-0058-4e2e-990a-c78d6b598601 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:43.315740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:43.316323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:43.316629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:43.316898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:43.388497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:43.389957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:43.493081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:43.507960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 561/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:43.517580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:43.586877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:43.623525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01057332-3049-4ed4-8f14-59e9bead3d61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:43.662392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:43.662945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:43.663200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:43.663469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] PATH_INFO: `/v3/projects/8ab699acf11447e89a9150a40c5b6864` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:43.664631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab699acf11447e89a9150a40c5b6864)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:43.666070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df132d5c-2377-423c-90bf-0d0a7e6f92d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:43.667692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 584/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:43 2022] GET /identity/v3/projects/8ab699acf11447e89a9150a40c5b6864 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:43.674529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:43.793456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695f47-71db-4d1b-bf57-8ba539e7fb8a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:43.830901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:43.831540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:43.831815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:43.832083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:43.832833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=8ab699acf11447e89a9150a40c5b6864, user.id=930e053bc85341188753cb9f16f5c90b) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:43.833134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:43.834561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5983-7ef0-41e9-962b-750c1a8c30cc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:43.840016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 562/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:43 2022] GET /identity/v3/role_assignments?user.id=930e053bc85341188753cb9f16f5c90b&scope.project.id=8ab699acf11447e89a9150a40c5b6864&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:45.671825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566703fa-12bc-4334-b9ae-f4b21aebc903 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:45.672372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566703fa-12bc-4334-b9ae-f4b21aebc903 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:45.672372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566703fa-12bc-4334-b9ae-f4b21aebc903 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:45.673733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 585/1147] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:45.679956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:45.680723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:45.680876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:45.709026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:45.710204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:45.764340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:45.786146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 563/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:46.049796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566703fa-12bc-4334-b9ae-f4b21aebc903 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:46.166628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566703fa-12bc-4334-b9ae-f4b21aebc903 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:46.201716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:46.202334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:46.202608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:46.202876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:46.204928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:46.214276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:46.219143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:46.220183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:46.221077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 586/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:46.226501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:46.327307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952d33-b22e-4c6a-81de-70c7f535dcf8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:46.362586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:46.363180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:46.363469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:46.363729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:46.364404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:46.364736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:46.366123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:46.374314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 564/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:46.380189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:46.484726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f05b0-dbe3-4c58-ba65-b0a94ff199c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:46.520357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:46.520984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:46.521255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:46.521524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] PATH_INFO: `/v3/users/930e053bc85341188753cb9f16f5c90b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:46.522760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] RBAC: Authorizing `identity:get_user(user_id=930e053bc85341188753cb9f16f5c90b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:46.524204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:46.525805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 587/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:46 2022] GET /identity/v3/users/930e053bc85341188753cb9f16f5c90b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:46.534765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:46.592626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:46.644857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b14ad-12b0-41f1-8c05-a6c181404765 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:46.691779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:46.692355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:46.692705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:46.692976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] PATH_INFO: `/v3/projects/8ab699acf11447e89a9150a40c5b6864` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:46.694243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] RBAC: Authorizing `identity:get_project(project_id=8ab699acf11447e89a9150a40c5b6864)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:46.695703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c6ce-486c-42e7-8be6-709c0218997d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:46.697389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 565/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:46 2022] GET /identity/v3/projects/8ab699acf11447e89a9150a40c5b6864 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:46.703502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:46.764077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:46.805121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969732a-6907-42c5-a0b3-1978864df742 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:46.843226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:46.843805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:46.844082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:46.844344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] PATH_INFO: `/v3/projects/8ab699acf11447e89a9150a40c5b6864/users/930e053bc85341188753cb9f16f5c90b/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:46.846599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab699acf11447e89a9150a40c5b6864, user_id=930e053bc85341188753cb9f16f5c90b, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:46.848108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f59cd-de5f-4683-b01d-1053ab9b717a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:46.867936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 588/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:04:46 2022] PUT /identity/v3/projects/8ab699acf11447e89a9150a40c5b6864/users/930e053bc85341188753cb9f16f5c90b/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:04:48.723745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7ea-b5d0-404f-864f-b05f195bebb7 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:48.724623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7ea-b5d0-404f-864f-b05f195bebb7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:48.724623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7ea-b5d0-404f-864f-b05f195bebb7 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:48.726878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 566/1154] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:48.733115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:48.733792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:48.733976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:48.755702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:48.756947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:48.831016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 589/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:49.099701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b7ea-b5d0-404f-864f-b05f195bebb7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:49.234031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b7ea-b5d0-404f-864f-b05f195bebb7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:49.252480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:49.277893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:49.278518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:49.278800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:49.279071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:49.290131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:04:49.294973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:49.296543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:04:49.297407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 567/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:49.302765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:49.367843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:49.418003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0e9d-d7b5-4a6e-806c-9742d3efb89c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:49.455539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:49.456218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:49.456497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:49.456787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:49.457453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:49.457733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:49.459121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:49.467302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 590/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:49.475859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:49.581432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b619f-f2d0-4fb1-aacd-f2e781dbf524 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:49.619304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:49.619867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:49.620136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:49.620406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] PATH_INFO: `/v3/users/930e053bc85341188753cb9f16f5c90b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:49.621680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] RBAC: Authorizing `identity:get_user(user_id=930e053bc85341188753cb9f16f5c90b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:49.623131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:49.624759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 568/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:04:49 2022] GET /identity/v3/users/930e053bc85341188753cb9f16f5c90b => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:49.631018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:49.733773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def29d66-3736-4ba2-ab96-352f5ea334c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:49.752088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:49.773359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:49.773957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:49.774231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:49.774494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:49.840604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:49.842047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:49.942271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 591/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:49.949171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:50.000754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:50.057881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be647b34-59eb-4f1b-a38c-12ef1d95182d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:50.091645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:50.092255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:50.092541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:50.092832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] PATH_INFO: `/v3/projects/8ab699acf11447e89a9150a40c5b6864` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:50.093969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab699acf11447e89a9150a40c5b6864)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:50.095397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e63843-45af-4796-b8c0-09c49a7a4c29 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:50.097051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 569/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:04:49 2022] GET /identity/v3/projects/8ab699acf11447e89a9150a40c5b6864 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:50.103410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:50.119235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:50.206056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cab431-cffd-42fe-bdc5-fea56a01d3d9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:50.241220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:50.241801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:50.242146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:50.242412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:50.243127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=8ab699acf11447e89a9150a40c5b6864, user.id=930e053bc85341188753cb9f16f5c90b) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:50.243468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:50.244919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87e2e-d876-4b0b-917a-bab59e33909a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:50.250658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 592/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:04:50 2022] GET /identity/v3/role_assignments?user.id=930e053bc85341188753cb9f16f5c90b&scope.project.id=8ab699acf11447e89a9150a40c5b6864&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:04:52.154825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3a298-59f9-4ea8-a0ff-79851c7c566c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:52.155756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3a298-59f9-4ea8-a0ff-79851c7c566c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:52.155756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3a298-59f9-4ea8-a0ff-79851c7c566c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:52.158003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 570/1162] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:52.164539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:52.165213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:52.165354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:52.194223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:52.195421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:52.263871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 593/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:52.531240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3a298-59f9-4ea8-a0ff-79851c7c566c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:52.652057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3a298-59f9-4ea8-a0ff-79851c7c566c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:52.665566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:52.688550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:52.689182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:52.689454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:52.689723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:52.757101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:52.758538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:52.858224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 571/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:04:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:04:52.865277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:52.880542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:52.971959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbac08-e8db-4d0f-bc53-805ddc018d56 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:53.006936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:53.007547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:53.007817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:53.008082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:53.014934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:53.016381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9a06c52b-cc91-41cf-bb0c-360c7aa13557 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:04:53.028178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:04:53.029319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:04:53.030183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 594/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:04:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:04:53.037397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:53.052322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:53.137196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab181c-2081-470b-af64-e14ca3631ebb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:53.172918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:53.173508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:53.173773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:53.174039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:53.174678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:04:53.175019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:53.176383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6981f9-3281-4c84-a417-136c55c137c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:53.183620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 572/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Aug 28 10:04:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:55.103929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da7fca-7644-4b29-ba31-67cbec69f68f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:55.104844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da7fca-7644-4b29-ba31-67cbec69f68f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:55.104844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da7fca-7644-4b29-ba31-67cbec69f68f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:55.106995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 595/1167] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:55.112447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49ba70-1dc7-4d53-a459-6a1c7729d562 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:55.112890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49ba70-1dc7-4d53-a459-6a1c7729d562 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:55.112998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49ba70-1dc7-4d53-a459-6a1c7729d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:55.132483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5c49ba70-1dc7-4d53-a459-6a1c7729d562 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:55.133687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5c49ba70-1dc7-4d53-a459-6a1c7729d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:55.204888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 573/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:55.472465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da7fca-7644-4b29-ba31-67cbec69f68f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:55.563911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:55.600273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da7fca-7644-4b29-ba31-67cbec69f68f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:55.635947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:55.636551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:55.636855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:55.637120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:55.637791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:55.639180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fed30-e62e-4dae-b152-0e78df731234 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:55.659610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 596/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Sun Aug 28 10:04:55 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:04:57.621622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217835-a987-4677-8a1b-78c0d8199911 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:57.622324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217835-a987-4677-8a1b-78c0d8199911 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:57.622582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217835-a987-4677-8a1b-78c0d8199911 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:57.624530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 574/1170] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:04:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:04:57.629905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:57.630332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:57.630420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:57.649734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:04:57.650905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:04:57.721989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 597/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:04:57.992070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2217835-a987-4677-8a1b-78c0d8199911 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:58.005870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:58.106088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2217835-a987-4677-8a1b-78c0d8199911 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:58.140374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:58.141018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:58.141286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:58.141553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] PATH_INFO: `/v3/domains/40cf2afa75f24859b8d5f7f5d9b723c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:58.153274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=40cf2afa75f24859b8d5f7f5d9b723c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:58.154709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32127ae-6b1b-47f8-8ae0-90e224bd9aba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:58.156286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 575/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Aug 28 10:04:57 2022] GET /identity/v3/domains/40cf2afa75f24859b8d5f7f5d9b723c2 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:04:58.162529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:04:58.223069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:04:58.271277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29437b09-9b62-4218-bf13-52f009cc1626 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:04:58.308174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:04:58.308794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:04:58.309068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:04:58.309330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:04:58.309983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:04:58.311423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bc0a0-e238-46b2-b357-83c9e18ae11b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:04:58.343390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 598/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:04:58 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:05:00.248286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528212-8934-4120-98cb-c3e60a30a2a2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:00.248888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528212-8934-4120-98cb-c3e60a30a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:00.248888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24528212-8934-4120-98cb-c3e60a30a2a2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:00.250963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 576/1174] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:05:00.257556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:00.258005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:00.258093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:00.278000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:05:00.279174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:05:00.347526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 599/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:05:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 28 10:05:00.614603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24528212-8934-4120-98cb-c3e60a30a2a2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:00.691291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:00.733484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24528212-8934-4120-98cb-c3e60a30a2a2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:00.768735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:00.769349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:00.769624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:00.769892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] PATH_INFO: `/v3/domains/40cf2afa75f24859b8d5f7f5d9b723c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:00.771123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=40cf2afa75f24859b8d5f7f5d9b723c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:00.772551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86157eb2-7422-4ad3-9a6b-4f23533b1f76 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:00.774131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 577/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Aug 28 10:05:00 2022] GET /identity/v3/domains/40cf2afa75f24859b8d5f7f5d9b723c2 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:00.780687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:00.823618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:00.888021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0fba5-fa39-43dc-8384-b7b120a0a2b8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:00.926162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:00.926767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:00.927042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:00.927308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:00.927959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:00.929445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1146dd-699b-44ad-83ff-304500f861f5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:00.956269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 600/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:05:00 2022] POST /identity/v3/users => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:05:02.866824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f158-5d5b-41c0-bf27-e180ddc3c5b2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:02.867736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f158-5d5b-41c0-bf27-e180ddc3c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:02.867736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f158-5d5b-41c0-bf27-e180ddc3c5b2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:02.869969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 578/1178] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:05:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:05:02.879007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:02.879439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:02.879544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:02.903733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:05:02.904973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:05:02.975501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 601/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:05:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:05:03.239105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32f158-5d5b-41c0-bf27-e180ddc3c5b2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:03.358328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32f158-5d5b-41c0-bf27-e180ddc3c5b2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:03.393272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:03.393895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:03.394165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:03.394438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:03.396401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:03.406147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:05:03.410984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:05:03.412024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:05:03.413268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 579/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:05:03.418145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:03.447465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:03.522114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8e4c-6dc5-4bc3-8717-f3d6d8f63f74 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:03.556815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:03.557414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:03.557686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:03.557950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:03.558611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:05:03.558911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:03.560293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:03.568400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 602/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:03.575019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:03.681997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24fe8-36b2-421c-816e-75a408ae90bb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:03.716376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:03.717016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:03.717306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:03.717562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] PATH_INFO: `/v3/users/ed3e797193a54208a6165099ad6d3fca` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:03.731505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e797193a54208a6165099ad6d3fca)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:03.732989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:03.734594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 580/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:05:03 2022] GET /identity/v3/users/ed3e797193a54208a6165099ad6d3fca => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:03.740866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:03.836566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:03.849221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cdfcb-082f-466e-8c91-75b52b7dac32 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:03.892176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:03.892836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:03.893173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:03.893514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:03.965415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:03.966871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:04.058723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:04.073640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 603/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:05:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:05:04.079853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:04.132977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:04.189914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3d9d-706d-4043-ba04-47a02b4ec3b4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:04.224557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:04.225194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:04.225467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:04.225732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] PATH_INFO: `/v3/projects/0b554510a3a942a58dca47be1966afc2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:04.226883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] RBAC: Authorizing `identity:get_project(project_id=0b554510a3a942a58dca47be1966afc2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:04.228301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1c771-6481-41bd-bf39-5f9aaf4c3d6a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:04.229963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 581/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:05:04 2022] GET /identity/v3/projects/0b554510a3a942a58dca47be1966afc2 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:04.236320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:04.341985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27c91d-2bf2-4a35-8126-3358a31491ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:04.388819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:04.389590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:04.389985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:04.390363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:04.391473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=0b554510a3a942a58dca47be1966afc2, user.id=ed3e797193a54208a6165099ad6d3fca) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:05:04.391908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:04.394073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cde6c-48b0-48c3-8fc4-55a462dbb78a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:04.399778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 604/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:05:04 2022] GET /identity/v3/role_assignments?user.id=ed3e797193a54208a6165099ad6d3fca&scope.project.id=0b554510a3a942a58dca47be1966afc2&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:06.298976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a444-06ce-4ae3-ba5e-eccd71ef7c17 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:06.299909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a444-06ce-4ae3-ba5e-eccd71ef7c17 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:06.299909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1a444-06ce-4ae3-ba5e-eccd71ef7c17 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:06.302024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 582/1186] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:05:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:05:06.308484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:06.309268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:06.309411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:06.332340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:05:06.333566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:05:06.385924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:06.412158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 605/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:05:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:05:06.684456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1a444-06ce-4ae3-ba5e-eccd71ef7c17 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:06.809487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1a444-06ce-4ae3-ba5e-eccd71ef7c17 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:06.832234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:06.849757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:06.850295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:06.850560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:06.850871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:06.861623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:05:06.866454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:05:06.867543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:05:06.868462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 583/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:05:06.874447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:06.986933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001ed74-ba72-4045-ac3c-9bad1c1be5a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:07.035879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:07.036531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:07.036811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:07.037079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:07.037752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:05:07.038051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:07.039448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:07.047923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 606/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:07.056707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:07.169025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef56930-d340-4bc4-b7b5-76c57963a6db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:07.213863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:07.214491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:07.214771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:07.215041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] PATH_INFO: `/v3/users/ed3e797193a54208a6165099ad6d3fca` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:07.216304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e797193a54208a6165099ad6d3fca)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:07.217893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:07.219493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 584/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:05:07 2022] GET /identity/v3/users/ed3e797193a54208a6165099ad6d3fca => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:07.227671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:07.279054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:07.343936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac9ad5-85d5-43a8-9418-de219cfc9a96 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:07.378844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:07.379459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:07.379730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:07.379995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] PATH_INFO: `/v3/projects/0b554510a3a942a58dca47be1966afc2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:07.381308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] RBAC: Authorizing `identity:get_project(project_id=0b554510a3a942a58dca47be1966afc2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:07.382742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d37ddb-d832-4585-801b-27d166c166b6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:07.384400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 607/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:05:07 2022] GET /identity/v3/projects/0b554510a3a942a58dca47be1966afc2 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:07.390992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:07.441060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:07.499220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055d65-934f-49aa-9078-deaa02e07566 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:07.535492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:07.536135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:07.536432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:07.536730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] PATH_INFO: `/v3/projects/0b554510a3a942a58dca47be1966afc2/users/ed3e797193a54208a6165099ad6d3fca/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:07.538866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b554510a3a942a58dca47be1966afc2, user_id=ed3e797193a54208a6165099ad6d3fca, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:07.540290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f44f31-f9c0-4d71-8d2c-07ef79de37d2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:07.556234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 585/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:05:07 2022] PUT /identity/v3/projects/0b554510a3a942a58dca47be1966afc2/users/ed3e797193a54208a6165099ad6d3fca/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:05:09.425150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9f8f-ae1b-4fee-a34b-b42b7e35092e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:09.426053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9f8f-ae1b-4fee-a34b-b42b7e35092e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:09.426053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb9f8f-ae1b-4fee-a34b-b42b7e35092e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:09.428199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 608/1193] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:05:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:05:09.433712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:09.434148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:09.434255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:09.453999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:05:09.455172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:05:09.523185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 586/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:05:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:05:09.823397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb9f8f-ae1b-4fee-a34b-b42b7e35092e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:09.952887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb9f8f-ae1b-4fee-a34b-b42b7e35092e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:09.966704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:09.988580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:09.989194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:09.989470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:09.989737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.000256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:05:10.005183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:05:10.006224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:05:10.007208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 609/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:05:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:05:10.012527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:10.054020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:10.137671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98057ca-a873-4bf8-a6f5-f29384ce47f7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:10.174494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:10.175104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:10.175384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:10.175657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.176332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:05:10.176666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:10.178061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:10.186103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 587/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:10.192410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:10.294560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b56303-bd53-4a06-99c1-0775c62149b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:10.333810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:10.334399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:10.334702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:10.334970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] PATH_INFO: `/v3/users/ed3e797193a54208a6165099ad6d3fca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.336208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e797193a54208a6165099ad6d3fca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:10.337702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:10.339258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 610/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Aug 28 10:05:10 2022] GET /identity/v3/users/ed3e797193a54208a6165099ad6d3fca => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:10.345300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:10.447024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:10.450041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03f961-1be3-4965-a950-b569be1d66a4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:10.485847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:10.486446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:10.486725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:10.487000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.558280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:10.560545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:10.659719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:10.665589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 588/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:05:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:05:10.672295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:10.717399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:10.781832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f19e-6187-4fdc-84d9-d6ae0a8f69aa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:10.817180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:10.817789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:10.818069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:10.818333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] PATH_INFO: `/v3/projects/0b554510a3a942a58dca47be1966afc2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.819568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] RBAC: Authorizing `identity:get_project(project_id=0b554510a3a942a58dca47be1966afc2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:10.821046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160973ef-236a-4316-bf88-b21ebf63b8e0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:10.822677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 611/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Aug 28 10:05:10 2022] GET /identity/v3/projects/0b554510a3a942a58dca47be1966afc2 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:05:10.832215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:05:10.936267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bc7b1-6ca9-40f8-b195-b2bea854faea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:05:10.971654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:05:10.972178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:05:10.972418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:05:10.972803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:05:10.973519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=0b554510a3a942a58dca47be1966afc2, user.id=ed3e797193a54208a6165099ad6d3fca) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:05:10.973822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:05:10.975231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7879d5d-60e3-434b-8705-7c78b2188328 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:05:10.981053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 589/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:05:10 2022] GET /identity/v3/role_assignments?user.id=ed3e797193a54208a6165099ad6d3fca&scope.project.id=0b554510a3a942a58dca47be1966afc2&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:05:20.812057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=46 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:05:20.978521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=68 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:05.833061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e7ef-6f22-4d18-9f34-47cb2a053f2e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:05.833714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e7ef-6f22-4d18-9f34-47cb2a053f2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:05.833714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e7ef-6f22-4d18-9f34-47cb2a053f2e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:05.835148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 612/1201] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:05.840819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:05.841257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:05.841476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:05.862658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:05.863906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:05.944702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 590/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:06.228095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7e7ef-6f22-4d18-9f34-47cb2a053f2e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:06.358937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7e7ef-6f22-4d18-9f34-47cb2a053f2e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:06.407240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:06.407982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:06.408294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:06.408653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:06.421439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:06.423311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:06.440301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:06.441587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:06.442564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:06.442564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:06.442564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:06.442564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 613/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:06.449724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:06.573259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d428df-a4e6-496c-84e0-87f3ee7e324c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:06.613863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:06.622316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:06.622992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:06.623229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:06.623490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:06.624134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:06.624430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:06.625848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acacb51-6d99-4fe5-80d0-6c98ef6d92ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:06.640547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 591/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:06.645028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:06.709700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:06.775787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044b4a4-3ff3-415a-b848-f4a7dc98160d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:06.813648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:06.814270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:06.814548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:06.814826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:06.815530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:06.817112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0944-af9c-4e38-9c64-b32c4414cc5c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:06.855406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 614/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Aug 28 10:06:06 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:06:08.800703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411829-3940-40dd-89a7-de039a7bcd64 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:08.801273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411829-3940-40dd-89a7-de039a7bcd64 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:08.801273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411829-3940-40dd-89a7-de039a7bcd64 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:08.802872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 592/1206] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:08.808586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:08.809029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:08.809137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:08.827576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:08.828794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:08.899928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 615/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:09.186061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75411829-3940-40dd-89a7-de039a7bcd64 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:09.302876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75411829-3940-40dd-89a7-de039a7bcd64 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:09.343791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:09.344606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:09.344773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:09.345047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:09.355790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:09.360778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:09.361800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:09.362666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:09.368552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 593/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:06:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:09.370902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:09.418257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:09.484791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd30c9-e63b-43f8-aa63-1d72635e61aa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:09.525564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:09.526193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:09.526474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:09.526746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:09.527427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:09.527741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:09.529256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:09.538967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 616/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:09.544343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:09.609483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:09.678037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2a586-1931-47ea-b54e-45cd63fe8fde None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:09.719471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:09.721660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:09.721660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:09.721660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:09.732403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:09.733988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:09.749983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:09.750992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:09.752457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:09.752457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:09.752457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:09.752457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 594/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:09.757215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:09.862829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:09.867278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c48660-60e4-4e94-be93-d70cd22b2470 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:09.906201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:09.906828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:09.907101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:09.907368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:09.908041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:09.908370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:09.909823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:09.930187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 617/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:09.932063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:10.047850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01350683-d1b1-4005-a387-a6e6d8cfbde1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:10.049942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:10.095778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:10.098937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:10.099250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:10.099593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:10.108269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:10.110156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:10.123509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:10.125650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:10.126623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 595/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:10.132083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:10.256299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e75e8-ab67-4e16-9386-7d49203e1b9b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:10.295006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:10.295763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:10.296061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:10.296398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:10.297579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:10.298000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:10.299542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:10.322047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 618/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:10.325954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:10.435653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2039b5-6d96-4319-8b2e-ab26f0022b98 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:10.473830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:10.474444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:10.474720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:10.474989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:10.513261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:10.557666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:10.559117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:10.679241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 596/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:06:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 28 10:06:10.685626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:10.706743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:10.795444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0694d6-ea81-4345-93b0-252602538d74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:10.834583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:10.835225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:10.835533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:10.835800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:10.847141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:10.848780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:10.865463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:10.872170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:10.873599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:10.873599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:10.873599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:10.873599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 619/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:10.874426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:10.943543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:10.990711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57a1d9-3df3-4700-81eb-eba9d3322d5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:11.031364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:11.032171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:11.032430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:11.032755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:11.033549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:11.034114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:11.035778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:11.049398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 597/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:11.055444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:11.135208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:11.171773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44997c92-157a-47d1-8452-ede3f6d5ffff None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:11.208269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:11.209088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:11.209623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:11.210027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:11.217679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:11.219590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:11.232851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:11.234497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:11.235392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 620/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:11.240235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:11.357224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4e-9ab5-491e-8351-21356d8999ed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:11.384453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:11.398295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:11.398919 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:11.399237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:11.399536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:11.400335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:11.400676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:11.402273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb41b9-c759-4a6a-974c-ac33743153a9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:11.416128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 598/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:06:11.422555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:11.536639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a647ec-4e26-4e6e-8283-e6741339f9d3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:11.569196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:11.575835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:11.576963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:11.576963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:11.577293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:11.578121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=ca82b32794ef4e37b9c2608a6e78930f) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:11.581182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:11.581182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bef476-3ef5-4cc7-8034-a977a9689b42 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:11.596535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 621/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:06:11 2022] GET /identity/v3/role_assignments?user.id=ca82b32794ef4e37b9c2608a6e78930f&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:13.467065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc014-379d-4956-bb6c-8f687ec10fad None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:13.467648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc014-379d-4956-bb6c-8f687ec10fad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:13.467648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc014-379d-4956-bb6c-8f687ec10fad None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:13.468959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 599/1220] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:13.475311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:13.475781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:13.476119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:13.499832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:13.501088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:13.572760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 622/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:13.861925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbc014-379d-4956-bb6c-8f687ec10fad None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:13.975139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbc014-379d-4956-bb6c-8f687ec10fad None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.011735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.012483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.012789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.013058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.024523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:14.029363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:14.030419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:14.031256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 600/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:06:14.036915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:14.149058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958ebc-caa2-4eec-94d6-041617d09ebc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.175756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:14.188973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.189738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.190024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.190391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.191350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:14.191806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:14.193622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:14.202770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 623/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:14.208730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:14.259210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:14.324256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce38f7-6386-4ca2-8239-b27881fce8c5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.361036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.361669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.361946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.362237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.373111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:14.374559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:14.390100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:14.391084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:14.392571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:14.392571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:14.392571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:14.392571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 601/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:14.397303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:14.507438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1e65e-e029-453f-952a-c84b0d967df4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.543702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.544353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.544676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.545083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.545592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:14.545911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:14.547683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:14.560570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 624/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:14.567236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:14.665897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:14.674443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae64eb7-d5f0-4bd7-83ac-df35f0059618 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.709891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.710518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.710793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.711060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.718466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:14.719947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:14.731321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:14.732458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:14.733627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:14.737173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 602/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:06:14.738671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:14.791195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:14.848971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870eece-913b-4f65-906e-1104df9a4a74 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:14.889726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:14.890507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:14.890797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:14.891126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:14.892019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:14.892317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:14.893845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:14.911853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 625/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:14.917825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:15.021629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded363e8-541b-4708-aa9a-a5697cc5fa89 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:15.062729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:15.063318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:15.063649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:15.063976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:15.077086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:15.078562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:15.093770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:15.094881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:15.095905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:15.095905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:15.095905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:15.095905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 603/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:15.100983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:15.205033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962efe0a-58ad-4b87-b2b0-addd6d73ef51 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:15.207954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:15.246060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:15.246823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:15.247124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:15.247436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:15.248079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:15.248385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:15.250236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:15.263587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 626/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:15.269989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:15.279734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:15.376394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa297-19a2-4d9d-a08f-40cb0c403c59 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:15.410269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:15.410866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:15.411118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:15.411389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:15.418195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:15.419679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:15.431310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:15.432210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:15.433451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 604/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:15.438521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:15.543502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01ca6-68f4-4ba6-9e3c-47db70842e3c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:15.583458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:15.584142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:15.584421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:15.584654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:15.585341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:15.585690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:15.587102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a4e47-f624-45a7-a726-fa3550d6aa07 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:15.600215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 627/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:15.606489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:15.685220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:15.725108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaaf4e-3066-42e2-aa7d-4aef1d150f4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:15.762850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:15.763491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:15.763750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:15.764016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/ca82b32794ef4e37b9c2608a6e78930f/roles/c1a4de0886a04a908142a262ce94b79c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:15.780039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=ca82b32794ef4e37b9c2608a6e78930f, role_id=c1a4de0886a04a908142a262ce94b79c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:15.781554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7048a-61ab-4872-a622-e33e11cb771a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:15.795138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 605/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:06:15 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/ca82b32794ef4e37b9c2608a6e78930f/roles/c1a4de0886a04a908142a262ce94b79c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:06:17.781851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ebbd-3146-4b33-abd1-950b9b08d56f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:17.782459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ebbd-3146-4b33-abd1-950b9b08d56f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:17.782459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ebbd-3146-4b33-abd1-950b9b08d56f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:17.783741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 628/1233] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:17.789400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:17.789843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:17.790044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:17.811605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:17.812962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:17.887836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 606/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:18.180012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ebbd-3146-4b33-abd1-950b9b08d56f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:18.205287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:18.287181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ebbd-3146-4b33-abd1-950b9b08d56f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:18.324831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:18.325543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:18.325819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:18.326084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:18.337183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:18.342145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:18.343132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:18.343943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 629/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Aug 28 10:06:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:18.350205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:18.403761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:18.464966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388267d-46e4-4dac-9c35-9763d8241b35 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:18.500418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:18.501060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:18.501342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:18.501607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:18.502276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:18.502573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:18.503968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:18.512312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 607/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Aug 28 10:06:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:18.518053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:18.614556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:18.631214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ebcfb-480b-49bc-be88-72aa7e9da6c9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:18.667546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:18.668177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:18.668447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:18.668748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:18.678843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:18.680293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:18.694888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:18.696015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:18.697535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:18.697535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:18.697535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:18.697535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 630/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:18.702432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:18.807465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d9a06-df15-408f-973e-d6d5c0a8d00c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:18.816742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:18.842739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:18.843355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:18.843640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:18.843967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:18.844617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:18.844916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:18.846304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:18.858711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 608/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:18.864801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:18.980369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf4ecc-1562-49c5-8f38-10dd968caf4c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:19.014280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:19.015013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:19.015354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:19.015683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:19.018515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:19.024612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:19.026063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:19.037148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:19.038254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:19.039459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 631/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:19.044622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:19.151492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03731f5-b72c-427e-ae76-9462bb49e3e1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:19.188188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:19.188826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:19.189111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:19.189384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:19.190165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:19.190493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:19.192036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:19.211581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 609/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:19.218116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:19.323441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abfb41-4f4e-4c80-bdb8-f1cff4fbdc2e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:19.359693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:19.360323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:19.360614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:19.360879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:19.423643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:19.430896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:19.432414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:19.536083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 632/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:06:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:06:19.543703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:19.572333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:19.649243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad1435-d56a-4ebc-a8d0-8ebd96348f2f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:19.684312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:19.684963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:19.685239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:19.685505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:19.697480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:19.698922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:19.715820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:19.721674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:19.722624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:19.722624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:19.722624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:19.722624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 610/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:19.722913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:19.824914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:19.836785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e646a-32fb-4dad-9fec-4229fb07a6bc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:19.872407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:19.873166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:19.873449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:19.873776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:19.874496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:19.874792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:19.876176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:19.892551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 633/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:19.894529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:19.977556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:20.003286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048b07-a366-4c28-bfa6-fe9699a34e69 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:20.039251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:20.039879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:20.040154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:20.040417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:20.047607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:20.049146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:20.061382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:20.062373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:20.063613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 611/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:20.068606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:20.177250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70649fce-5a3c-45f0-9d64-949e9862c39a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:20.212443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:20.213086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:20.213369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:20.213642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:20.214331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:20.214662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:20.216290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd118f-f6c8-4b7e-9477-33d4acf912f5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:20.230273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 634/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:20.236526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:20.339920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f618d7-4686-4c33-bc08-cab40bad501d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:20.378367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:20.378957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:20.379230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:20.379491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:20.380210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] RBAC: Adding query filter params (role.id=c1a4de0886a04a908142a262ce94b79c, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=ca82b32794ef4e37b9c2608a6e78930f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:20.380567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:20.382087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32667b-c1e9-4afa-8eec-4252338f9ee4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:20.389028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 612/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:06:20 2022] GET /identity/v3/role_assignments?user.id=ca82b32794ef4e37b9c2608a6e78930f&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=c1a4de0886a04a908142a262ce94b79c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:22.353226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d287e7-91fc-4dfc-afe1-0750adcb4699 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:22.353851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d287e7-91fc-4dfc-afe1-0750adcb4699 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:22.353851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d287e7-91fc-4dfc-afe1-0750adcb4699 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:22.355345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 635/1247] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:22.361376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:22.361815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:22.361897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:22.369116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:22.383309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:22.384576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:22.459173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 613/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:22.733021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d287e7-91fc-4dfc-afe1-0750adcb4699 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:22.755072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:22.845023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d287e7-91fc-4dfc-afe1-0750adcb4699 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:22.885263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:22.885894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:22.886172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:22.886439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:22.897255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:22.902059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:22.903018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:22.904104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 636/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:22.909399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.025911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ba10-85a2-473f-8f72-d94cd5dc8ad8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.059041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:23.066729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.067584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.068063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.068611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:23.069549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:23.070056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:23.072324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:23.086539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 614/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:23.093218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.186434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:23.212997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c35e4-ef8d-4337-9480-4a1f7598a445 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.252375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.253125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.253410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.253766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:23.264952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:23.266681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:23.284204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:23.285293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:23.286627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:23.286627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:23.286627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:23.286627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 637/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:23.291693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.402392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d5492-b1de-4138-b0c0-fb1265b77450 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.444749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.445490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.445769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.446078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:23.446767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:23.447074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:23.448454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:23.460666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 615/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:23.467117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.572111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436e0e-953a-4138-8cd9-208e58458bd2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.600758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:23.609142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.609728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.610001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.610262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:23.617907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:23.619390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:23.630848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:23.631999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:23.633132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 638/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:23.637907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.696294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:23.743936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f4cc5-24f0-4a09-beac-e05386332e13 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.778773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.779385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.779657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.779924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:23.780703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:23.781000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:23.782432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:23.800652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 616/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:23.806715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:23.918388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcd58b-2c2e-47e6-9e88-335d623c1cef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:23.958120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:23.958751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:23.959057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:23.959360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:24.019457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:24.032752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:24.034313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:24.148039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 639/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:06:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 28 10:06:24.153953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:24.277571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1a247-9a10-401f-91b0-00c36b476194 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:24.291409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:24.315087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:24.315760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:24.316039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:24.316305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:24.327508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:24.329016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:24.345330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:24.346646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:24.347600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:24.347600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:24.347600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:24.347600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 617/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:06:24.352630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:24.453518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:24.467475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511583-09bc-4ec8-8d8c-ea85d684c434 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:24.502839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:24.503459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:24.506263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:24.506613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:24.507241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:24.507543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:24.508975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:24.520780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 640/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:06:24.526691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:24.644700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b81a-c71e-49a7-9a6d-c506766afd4c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:24.685380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:24.686022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:24.686295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:24.686580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:24.693820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:24.695295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:24.706540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:24.707545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:24.709079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 618/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:24.713790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:24.822102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4898d0-b92c-467f-8939-35a04806319e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:24.858069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:24.858776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:24.859002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:24.859313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:24.860018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:24.860324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:24.861805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ce66f-bc3f-4d16-9db7-2d00f7f16b86 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:24.874585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 641/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:24.881662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:24.911709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:24.990399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f8c3-7425-45db-b6f2-865b6c852779 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:25.026389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:25.027097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:25.027467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:25.027736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:25.028706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=ca82b32794ef4e37b9c2608a6e78930f) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:25.029020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:25.030832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9788e-fc74-4f17-901e-6d44d7ed5cb3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:25.037097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 619/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:06:24 2022] GET /identity/v3/role_assignments?user.id=ca82b32794ef4e37b9c2608a6e78930f&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:26.980626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d8899-7877-4f7f-8102-c0e6ae95e9fc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:26.981246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d8899-7877-4f7f-8102-c0e6ae95e9fc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:26.981246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d8899-7877-4f7f-8102-c0e6ae95e9fc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:26.982505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 642/1261] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:26.988072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:26.988607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:26.988607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:27.007871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:27.009133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:27.082633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 620/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:27.342620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d8899-7877-4f7f-8102-c0e6ae95e9fc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:27.354134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:27.451274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d8899-7877-4f7f-8102-c0e6ae95e9fc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:27.485719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:27.486344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:27.486623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:27.486892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:27.497588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:27.502438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:27.503496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:27.504641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 643/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:27.512184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:27.533045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:27.619233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8cde-15c9-4cf3-942b-1f99ca5020ae None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:27.654193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:27.654799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:27.655079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:27.655348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:27.656020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:27.656327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:27.657745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:27.665905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 621/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:06:27.672072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:27.752750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:27.787659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536973cb-5866-442d-a35a-a5245e36fd61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:27.822642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:27.823255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:27.823552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:27.823800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:27.835397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:27.836889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:27.851309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:27.852354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:27.853377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:27.853377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:27.853377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:27.853377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 644/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:27.860216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:27.946888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:27.976265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878876aa-19a3-4d5f-ac4d-dd7064c87f4a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.010933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.011539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.011792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.012062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.012738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:28.013033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.014411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.026206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 622/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:06:28.035013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:28.142811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f498-290e-4ce2-a5a7-d13c8981902f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.162411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:28.185148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.186113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.186512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.186898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.196181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.197749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:28.209269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:28.210335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:28.211471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 645/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:28.216347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:28.322130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faaab5-e496-41f2-9c82-414f5b2769a4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.357450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.358050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.358315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.358596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.359324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:28.359630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.361093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.369782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:28.383932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 623/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:28.392314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:28.496985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413150c-541b-4597-9f70-8337a40326c2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.531900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.532542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.532809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.533076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.543809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.545288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:28.562044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:28.563061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:28.564132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:28.564132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:28.564132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:28.564132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 646/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 28 10:06:28.570142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:28.679149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd4c3a-12e3-4a94-9c04-10c403ff888a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.715007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.715608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.715887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.716236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.716936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:28.717237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.718614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.730255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 624/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:28.739420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:28.778144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:28.856532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5c06-d8af-4091-ad13-21cc2d4ce395 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:28.901300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:28.902026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:28.902300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:28.902614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:28.911833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:28.913504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:28.926369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:28.927383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:28.928402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 647/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:28.934214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:29.000444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:29.044439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d3c65-593e-4532-94f7-45fc479cad15 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:29.085834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:29.086446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:29.086728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:29.086997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:29.087701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:29.088011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:29.089514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348cab-a4c5-4c7d-aa11-ba874f734873 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:29.101830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 625/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:29.108403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:29.207785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:29.222146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092f7b4-5535-4cb6-beab-596d17dbc624 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:29.262668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:29.263308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:29.263578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:29.263843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] PATH_INFO: `/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/ca82b32794ef4e37b9c2608a6e78930f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:29.265788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f7fa10d292c4804a7550be0196a1b9f, user_id=ca82b32794ef4e37b9c2608a6e78930f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:29.267223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acce01-6c63-44bf-a0f3-308e37184530 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:29.283421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 648/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Aug 28 10:06:29 2022] PUT /identity/v3/projects/4f7fa10d292c4804a7550be0196a1b9f/users/ca82b32794ef4e37b9c2608a6e78930f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:06:31.146518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafc7e-e804-47c9-be0b-4450e238d3bd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:31.147077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafc7e-e804-47c9-be0b-4450e238d3bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:31.147077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafc7e-e804-47c9-be0b-4450e238d3bd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:31.148432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 626/1274] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:31.154743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:31.155442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:31.155644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:31.176342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:31.177558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:31.245448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 649/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:31.510651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cafc7e-e804-47c9-be0b-4450e238d3bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:31.626612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cafc7e-e804-47c9-be0b-4450e238d3bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:31.644037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:31.661169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:31.661793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:31.662069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:31.662334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:31.673273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 28 10:06:31.678078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:31.679144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:31.680062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 627/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Aug 28 10:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:31.685554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:31.769879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:31.790299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd44e6-1a04-4c73-a53f-553b970addb4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:31.825557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:31.826155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:31.826432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:31.826696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:31.827354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:31.827656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:31.829068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:31.838040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 650/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Aug 28 10:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:31.844589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:31.946269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9484-75b1-4c56-bbfa-5fd27a0f103a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:31.985817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:31.986438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:31.986715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:31.986983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:31.997699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:31.999165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:32.014940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:32.015980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:32.017232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:32.017232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:32.017232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:32.017232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 628/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:32.022404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:32.137988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3546a-0aee-40a3-8e97-51cc1eae02f7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:32.174368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:32.175012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:32.175270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:32.175533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:32.176161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:32.176452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:32.177850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:32.192328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 651/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:32.198460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:32.231943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:32.319882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a40dc0-6d02-44ae-a15f-32dfaad3fe5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:32.358538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:32.359147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:32.359419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:32.359689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:32.367096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:32.368539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 28 10:06:32.379679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:32.380808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:32.381773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 629/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:32.386814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:32.391117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:32.492142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0bdbb-c4da-4f24-bbc1-7c2f4455b619 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:32.530390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:32.530995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:32.531264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:32.531529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:32.532255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:32.532549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:32.534066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:32.551803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 652/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Aug 28 10:06:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:32.557979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:32.658724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:32.671194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896879d8-aa19-4401-b0d3-d5428afd3d85 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:32.710874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:32.711472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:32.711747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:32.712019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:32.789003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:32.791256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:32.890575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:32.906987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 630/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Aug 28 10:06:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 28 10:06:32.913854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:32.979416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:33.019421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5aee5-c637-4c34-a757-477a44950aec None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:33.055584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:33.056203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:33.056477 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:33.056777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:33.067224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:33.068711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 28 10:06:33.083676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:33.084810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 28 10:06:33.086336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 28 10:06:33.086336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 28 10:06:33.086336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:33.086336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 653/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:33.093864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:33.202883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59f9d-4d07-41da-be07-0273fdb78164 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:33.237856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:33.238449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:33.238730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:33.239002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:33.239648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:33.239949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:33.241356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:33.252583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 631/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Aug 28 10:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:33.261600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:33.381347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c520-c6b6-4765-a862-0d0cffc993d6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:33.398112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:33.424272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:33.424920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:33.425195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:33.425464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:33.432309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:33.433798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 28 10:06:33.445250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:33.446281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 28 10:06:33.447332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Aug 28 10:06:33.448664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 654/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Aug 28 10:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:33.452737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:33.511545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:33.566250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f70ac-7cef-42a8-8b1e-99f0ad1deff1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:33.603478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:33.604086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:33.604362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:33.604669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:33.605355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:33.605658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:33.607092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec83c48-16e8-4546-88b1-3f535b183d6e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:33.619101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 632/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Aug 28 10:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:33.625351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:33.727485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05240d-cd84-4d40-a258-71460c979f7f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:33.762445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:33.762955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:33.763230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:33.763498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:33.764212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] RBAC: Adding query filter params (role.id=4f6d8438189649f09ae3243d86ab3381, scope.project.id=4f7fa10d292c4804a7550be0196a1b9f, user.id=ca82b32794ef4e37b9c2608a6e78930f) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:33.764571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:33.765971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55747131-42b7-4b18-b58e-5dbc7158a480 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:33.771855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 655/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Aug 28 10:06:33 2022] GET /identity/v3/role_assignments?user.id=ca82b32794ef4e37b9c2608a6e78930f&scope.project.id=4f7fa10d292c4804a7550be0196a1b9f&role.id=4f6d8438189649f09ae3243d86ab3381 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:35.662561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1b0e-4044-4edc-9d10-b57ff85a1eb8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:35.663124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1b0e-4044-4edc-9d10-b57ff85a1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:35.663124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1b0e-4044-4edc-9d10-b57ff85a1eb8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:35.664464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 633/1288] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:35.670807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:35.671548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:35.671688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:35.692353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:35.693593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:35.724358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:35.763550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 656/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:36.032404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1b0e-4044-4edc-9d10-b57ff85a1eb8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:36.166959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1b0e-4044-4edc-9d10-b57ff85a1eb8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:36.177954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:36.203033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:36.203646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:36.203922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:36.204185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:36.204789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:36.206160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:36.214848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:36.216090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:36.217133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 634/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:06:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:36.222154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:36.319868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc10bc-e72c-419c-9b23-1f749a0858a4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:36.353862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:36.354458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:36.354727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:36.354987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:36.355635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:36.355911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:36.357308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe8d46-9a2d-40aa-9b27-7bf96b6746da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:36.361998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 657/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Aug 28 10:06:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:36.367918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:36.469368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824056bc-b136-4ebd-afc1-814ca2bec97c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:36.503265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:36.503878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:36.504169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:36.504441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:36.505091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:36.505387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:36.506762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a4024-9b8d-431e-8de2-83703226c873 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:36.511229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 635/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Aug 28 10:06:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:06:38.382961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ca44c-2255-4e64-b5b0-0fa0d6b99241 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:38.383559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ca44c-2255-4e64-b5b0-0fa0d6b99241 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:38.383559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ca44c-2255-4e64-b5b0-0fa0d6b99241 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:38.384880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 658/1293] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:38.391129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f541a1f-193f-4cec-9b41-8bf514a56d9a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:38.391841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f541a1f-193f-4cec-9b41-8bf514a56d9a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:38.391994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f541a1f-193f-4cec-9b41-8bf514a56d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:38.418270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3f541a1f-193f-4cec-9b41-8bf514a56d9a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:38.419472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3f541a1f-193f-4cec-9b41-8bf514a56d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:38.450990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:38.490375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 636/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:38.752850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ca44c-2255-4e64-b5b0-0fa0d6b99241 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:38.792811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:38.869285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ca44c-2255-4e64-b5b0-0fa0d6b99241 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:38.904654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:38.905250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:38.905525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:38.905789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:38.906437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:38.907822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffd2c1-912c-454c-bc70-4995be3e6d63 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:38.929425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 659/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Aug 28 10:06:38 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:06:40.822351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360886e-0685-48b7-bb99-d262535568f4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:40.823190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360886e-0685-48b7-bb99-d262535568f4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:40.823190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360886e-0685-48b7-bb99-d262535568f4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:40.825511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 637/1296] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:40.831760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:40.832483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:40.832667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:40.857741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:40.858898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:40.936247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 660/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:41.207646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360886e-0685-48b7-bb99-d262535568f4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:41.317714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360886e-0685-48b7-bb99-d262535568f4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:41.352384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:41.353013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:41.353288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:41.353556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:41.354133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:41.355490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:41.364370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:41.365515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:41.366579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 638/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:06:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:41.371469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:41.391485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:41.475410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c048da0-7560-4bf7-9cf9-2564c1b2a1aa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:41.513865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:41.514466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:41.514741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:41.515005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:41.515636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:41.515937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:41.517350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:41.522579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 661/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Aug 28 10:06:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:41.529222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:41.563575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:41.636022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d9110-2bc9-4297-8613-9fc6200cf60f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:41.670636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:41.671227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:41.671500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:41.671765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:41.672452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] RBAC: Adding query filter params (interface=public, service_id=974e523ac3f544b78a244a6f60b60d83, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:41.673135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:41.674494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742bae9-2ad7-432c-b038-d2274009c9b3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:41.679617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 639/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Aug 28 10:06:41 2022] GET /identity/v3/endpoints?service_id=974e523ac3f544b78a244a6f60b60d83&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:41.685671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:41.768589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:41.787328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b0dbf-282b-4988-b3dd-096c5474dac4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:41.822606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:41.823195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:41.823467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:41.823732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:41.824332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:41.825741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63889e85-ece9-40ff-a813-9eaaa598fae9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:41.832602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 662/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Aug 28 10:06:41 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:06:43.741513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425a4d9-e3e3-432d-8e03-ddb948112a12 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:43.742107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425a4d9-e3e3-432d-8e03-ddb948112a12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:43.742107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425a4d9-e3e3-432d-8e03-ddb948112a12 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:43.743416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 640/1302] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:06:43.749772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:43.750518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:43.750650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:43.771937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:06:43.773153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:06:43.844943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 663/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:06:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 28 10:06:44.124166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425a4d9-e3e3-432d-8e03-ddb948112a12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:44.231537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:44.242622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425a4d9-e3e3-432d-8e03-ddb948112a12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:44.278647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:44.279265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:44.279538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:44.279802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:44.280379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:44.281769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 28 10:06:44.291028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 28 10:06:44.292142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 28 10:06:44.293118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Aug 28 10:06:44.296716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 641/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Aug 28 10:06:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 28 10:06:44.299354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:44.389110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:44.413721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935d19-22d7-4d21-8d90-0abcc4a556dd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:44.449219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:44.449808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:44.450103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:44.450380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:44.450986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:06:44.451286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:44.452688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39273662-f3b8-4945-9783-58cab9d19720 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:44.458256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 664/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Aug 28 10:06:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:06:44.467030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:06:44.573553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28727c8-e4a1-46cc-8f84-6b76801861c5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:06:44.616558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:06:44.617202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:06:44.617483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:06:44.617759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:06:44.618424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:06:44.619806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d032254-88c0-4d00-8a68-20f6809bb933 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:06:44.649323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 642/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:06:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:06:54.454773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=30 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:06:54.641959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:48.109260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d1b45-74bc-40c8-ad98-ebb5484f9119 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:48.109868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d1b45-74bc-40c8-ad98-ebb5484f9119 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:48.109868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d1b45-74bc-40c8-ad98-ebb5484f9119 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:48.115879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 665/1307] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:07:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:48.117586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532b1-f6f5-451e-b750-85abdeb6a431 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:48.118491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532b1-f6f5-451e-b750-85abdeb6a431 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:48.118972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532b1-f6f5-451e-b750-85abdeb6a431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:48.210988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c3a532b1-f6f5-451e-b750-85abdeb6a431 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:48.212254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3a532b1-f6f5-451e-b750-85abdeb6a431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:48.337006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 643/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:07:48.631395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:48.631737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:48.632181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:48.634322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 666/1309] 10.222.0.40 () {54 vars in 965 bytes} [Sun Aug 28 10:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:48.640440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdbe2-4e40-4e65-830d-ad3b406b19b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:48.640915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdbe2-4e40-4e65-830d-ad3b406b19b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:48.641383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdbe2-4e40-4e65-830d-ad3b406b19b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:48.778167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-290bdbe2-4e40-4e65-830d-ad3b406b19b7 None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:48.779684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-290bdbe2-4e40-4e65-830d-ad3b406b19b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:48.805202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:48.873585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 644/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Aug 28 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:07:48.882418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:07:48.884420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:49.008712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf69c-26e2-409b-a98b-f9ecf8ad1f0f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:07:49.103459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:07:49.104487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:49.105333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:49.105821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:49.107748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:49.180991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:49.255152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:07:49.256663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbba30-58af-4240-9722-d2dc32dbaa7c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:07:49.279256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 667/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:07:51.518135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1e35b-51ea-4397-853e-514c0eea3187 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:51.518733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1e35b-51ea-4397-853e-514c0eea3187 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:51.518733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1e35b-51ea-4397-853e-514c0eea3187 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:51.520056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 645/1312] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:51.525662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355168-95e2-4503-a217-5da04d81db8a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:51.526103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355168-95e2-4503-a217-5da04d81db8a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:51.526257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355168-95e2-4503-a217-5da04d81db8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:51.611130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7f355168-95e2-4503-a217-5da04d81db8a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:51.612383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7f355168-95e2-4503-a217-5da04d81db8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:51.726054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 668/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:07:52.021671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:52.021964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:52.022250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:52.023570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 646/1314] 10.222.0.40 () {54 vars in 965 bytes} [Sun Aug 28 10:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:52.029844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b77-a2b1-4a2c-a2c6-d00c4df89e1c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:52.030153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b77-a2b1-4a2c-a2c6-d00c4df89e1c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:52.030420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b77-a2b1-4a2c-a2c6-d00c4df89e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:52.043908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:52.107903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-69ed7b77-a2b1-4a2c-a2c6-d00c4df89e1c None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:52.109129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-69ed7b77-a2b1-4a2c-a2c6-d00c4df89e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:52.191440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 669/1315] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Aug 28 10:07:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:07:52.201316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:07:52.203725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:52.310963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c879-9707-456a-bac9-c2c4d4ee5b5e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:07:52.386638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:52.389644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:07:52.390255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:52.390503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:52.391191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:52.393498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:52.507292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:07:52.508904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821bedf-a223-4cee-b7c4-427c5b027e38 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:07:52.529118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 647/1316] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:07:54.963704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:54.964695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:54.964695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:54.966759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 670/1317] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:07:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:54.974740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d150-2879-46aa-b28f-8739f6d026fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:54.975142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d150-2879-46aa-b28f-8739f6d026fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:54.975310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d150-2879-46aa-b28f-8739f6d026fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:55.047867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a18d150-2879-46aa-b28f-8739f6d026fa None None] MFA Rules not processed for user `b2c1bfd1c0e7420b9a8651065c3d719d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:55.049118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a18d150-2879-46aa-b28f-8739f6d026fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:55.105953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:55.152271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 648/1318] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:07:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:07:55.436698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:07:55.439176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:55.555543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157508dc-76e9-41e6-af4a-444e1740553f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:07:55.558306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:55.635431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:07:55.636045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:55.636324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:55.636609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:55.638215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:55.754778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:07:55.756213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf707f-a117-4b61-9e03-e36b1bde8a9f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:07:55.774784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 671/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:07:58.351158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:58.351689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:58.351689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:58.353032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 649/1320] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:07:58.359288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e7316-e49a-4a73-87d1-2a32e95fee65 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:58.360012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e7316-e49a-4a73-87d1-2a32e95fee65 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:58.360155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e7316-e49a-4a73-87d1-2a32e95fee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:58.402886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:58.430535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-888e7316-e49a-4a73-87d1-2a32e95fee65 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:07:58.431724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-888e7316-e49a-4a73-87d1-2a32e95fee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:58.536541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 672/1321] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:07:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:07:58.828227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:07:58.830693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:58.961842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cfbde-be6e-4e47-a6d9-2963092aa518 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:07:59.036862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:07:59.037473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:07:59.037752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:07:59.038023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:07:59.039630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:07:59.047029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:07:59.160300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:07:59.161814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5066b-ccbc-4a9a-bfd9-a5c0e31d5f88 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:07:59.185583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 650/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:01.774374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:01.774918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:01.774918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:01.776405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 673/1323] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:08:01.781847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3ed1a-c0b5-4ba8-a5a6-38cd0dff9403 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:01.782309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3ed1a-c0b5-4ba8-a5a6-38cd0dff9403 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:01.782309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3ed1a-c0b5-4ba8-a5a6-38cd0dff9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:01.848603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a7d3ed1a-c0b5-4ba8-a5a6-38cd0dff9403 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:01.849713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7d3ed1a-c0b5-4ba8-a5a6-38cd0dff9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:01.908472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:01.950853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 651/1324] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:02.230368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:02.231908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:02.339370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d934dd7-77dd-407a-8675-34247b770d07 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:02.351765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:02.415655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:02.416133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:02.416408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:02.416715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:02.418297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:02.529547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:02.531011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369535-d284-419f-b314-e4cdbe1db700 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:02.549562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 674/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:04.504551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:04.505120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:04.505120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:04.506574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 652/1326] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:04.511769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbac9-b53a-41ec-83c2-a05548c05e8d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:04.512201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbac9-b53a-41ec-83c2-a05548c05e8d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:04.512295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbac9-b53a-41ec-83c2-a05548c05e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:04.564473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:04.585099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-27afbac9-b53a-41ec-83c2-a05548c05e8d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:04.586438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27afbac9-b53a-41ec-83c2-a05548c05e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:04.686835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 675/1327] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:04.967307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:04.969395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:05.085168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88282316-25e0-4cf2-8c2e-72179b65f872 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:05.158279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:05.158860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:05.159164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:05.159429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:05.161055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:05.178453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:05.274401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:05.275749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfab0c-2d43-4773-bfdf-bc3e9b67dcf1 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:05.295617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 653/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:07.245391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:07.246013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:07.246013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:07.247266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 676/1329] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:07.252668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d74f-21de-46f5-98db-b417348489b5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:07.253107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d74f-21de-46f5-98db-b417348489b5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:07.253208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d74f-21de-46f5-98db-b417348489b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:07.323710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5434d74f-21de-46f5-98db-b417348489b5 None None] MFA Rules not processed for user `b2c1bfd1c0e7420b9a8651065c3d719d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:07.325000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5434d74f-21de-46f5-98db-b417348489b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:07.391336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:07.426321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 654/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:07.705033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:07.706583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:07.816734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ac20-8fbd-4b2d-a16a-a00c61a9a6c6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:07.839258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:07.898035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:07.898649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:07.898923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:07.899194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:07.900829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:08.016267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:08.017780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f161-b597-464b-81e3-97539bce617c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:08.036889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 677/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:10.221451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:10.222020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:10.222020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:10.223427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 655/1332] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:10.228846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2fbe4-c710-4b86-8cbe-dd0bcd4df54c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:10.229266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2fbe4-c710-4b86-8cbe-dd0bcd4df54c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:10.229374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2fbe4-c710-4b86-8cbe-dd0bcd4df54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:10.306341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:10.309304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-66e2fbe4-c710-4b86-8cbe-dd0bcd4df54c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:10.311225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-66e2fbe4-c710-4b86-8cbe-dd0bcd4df54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:10.412037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 678/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:10.697032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:10.699499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:10.816874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a008-296f-46f1-9a0f-af979e824eb4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:10.897420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:10.898023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:10.898294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:10.898557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:10.900133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:10.928152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:11.011595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:11.013079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b27e4b-b926-4a52-9048-ba30d66a45cf service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:11.031729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 656/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:12.961741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:12.962337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:12.962337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:12.963631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 679/1335] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:12.970064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3414f-c796-447e-9589-1f3ecafbedb6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:12.970757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3414f-c796-447e-9589-1f3ecafbedb6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:12.970984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3414f-c796-447e-9589-1f3ecafbedb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:13.043284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6da3414f-c796-447e-9589-1f3ecafbedb6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:13.044539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6da3414f-c796-447e-9589-1f3ecafbedb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:13.124458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:13.147256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 657/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:13.430792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:13.433277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:13.552405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151115-68f1-4e3e-b626-f333055b5a79 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:13.583641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:13.630460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:13.631055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:13.631327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:13.631596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:13.633218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:13.747862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:13.749249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e7b3-3837-4b4d-9a7a-b141dc431762 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:13.767668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 680/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:15.984611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:15.985205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:15.985205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:15.986546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 658/1338] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:15.992949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d5e1-4558-4e67-baba-a6c0abff05a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:15.993413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d5e1-4558-4e67-baba-a6c0abff05a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:15.993514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d5e1-4558-4e67-baba-a6c0abff05a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:16.066365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5747d5e1-4558-4e67-baba-a6c0abff05a4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:16.067620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5747d5e1-4558-4e67-baba-a6c0abff05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:16.075817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:16.170794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 681/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:16.456613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:16.458861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:16.568722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea3668-a2e0-4905-8555-a45114e78c65 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:16.650531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:16.651148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:16.651424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:16.651693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:16.653340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:16.689860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:16.772212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:16.773614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08237202-4028-4430-b245-3a8008df7458 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:16.793053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 659/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:20.680166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:20.680745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:20.680745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:20.682113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 682/1341] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:20.687551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e43caf-aa1b-4f18-8a54-1b36065349f6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:20.687995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e43caf-aa1b-4f18-8a54-1b36065349f6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:20.688109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e43caf-aa1b-4f18-8a54-1b36065349f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:20.756167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f3e43caf-aa1b-4f18-8a54-1b36065349f6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:20.757422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3e43caf-aa1b-4f18-8a54-1b36065349f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:20.968392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:20.982585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 660/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:21.259097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:21.260668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:21.370502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8e17-ebf4-48a8-86c5-6443f48b297f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:21.409667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:21.447726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:21.448338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:21.448675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:21.448941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:21.450569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:21.563228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:21.564650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc65f4f-6b45-4eb3-a204-7087db9e64d7 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:21.586453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 683/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:24.319242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:24.319772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:24.319772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:24.321034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 661/1344] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:24.326292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727339f0-17b8-433d-bfad-4f390899024c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:24.326733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727339f0-17b8-433d-bfad-4f390899024c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:24.326827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727339f0-17b8-433d-bfad-4f390899024c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:24.393838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-727339f0-17b8-433d-bfad-4f390899024c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:24.395048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-727339f0-17b8-433d-bfad-4f390899024c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:24.413796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:24.496988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 684/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:24.777934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:24.779481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:24.889421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a69f0c-203b-4ea2-af8e-da8c5ef4d3f2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:24.970149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:24.970770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:24.971044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:24.971284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:24.972977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:25.016973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:25.093688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:25.095226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda342-a05d-4abe-88e6-f67da20a27ff service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:25.115896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 662/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:28.954583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:28.955119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:28.955119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:28.956526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 685/1347] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:28.962568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a5c46-c468-4ede-8eb5-38ac7a679545 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:28.963061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a5c46-c468-4ede-8eb5-38ac7a679545 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:28.963507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a5c46-c468-4ede-8eb5-38ac7a679545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:29.036747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df0a5c46-c468-4ede-8eb5-38ac7a679545 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:29.037996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df0a5c46-c468-4ede-8eb5-38ac7a679545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:29.138930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:29.141697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 663/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:08:29.433786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:29.436385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:29.550334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d53b39-fa4b-49db-84a2-73f22297974f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:29.599568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:29.627799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:29.628412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:29.628801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:29.629074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:29.630702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:29.749019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:29.750466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03794bdc-719c-438c-9d51-55de58cf341e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:29.768385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 686/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:34.620333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad12e-6604-4b4b-a20d-1e865a806a4c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:34.620818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad12e-6604-4b4b-a20d-1e865a806a4c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:34.621068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ad12e-6604-4b4b-a20d-1e865a806a4c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:34.624469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 664/1350] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:34.630398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e513b-ce53-4e8b-bc21-363d7a68fb7c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:34.631236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e513b-ce53-4e8b-bc21-363d7a68fb7c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:34.631648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e513b-ce53-4e8b-bc21-363d7a68fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:34.712151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6a1e513b-ce53-4e8b-bc21-363d7a68fb7c None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:34.713476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6a1e513b-ce53-4e8b-bc21-363d7a68fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:34.732255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:34.876648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 687/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:38.988171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5883e-2dff-4eb3-bbe5-0b27803e78ed None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:38.988805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5883e-2dff-4eb3-bbe5-0b27803e78ed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:38.988805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5883e-2dff-4eb3-bbe5-0b27803e78ed None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:38.990444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 665/1352] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:38.995212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce89bc-2f36-4267-881c-10244d39acda None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:38.995700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce89bc-2f36-4267-881c-10244d39acda None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:38.995948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce89bc-2f36-4267-881c-10244d39acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:39.074901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f8ce89bc-2f36-4267-881c-10244d39acda None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:39.076108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f8ce89bc-2f36-4267-881c-10244d39acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:39.148605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:39.186517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 688/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:08:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:40.544557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488778-7d7a-4b64-bef9-f738fb7e12fc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:40.545150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488778-7d7a-4b64-bef9-f738fb7e12fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:40.545150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488778-7d7a-4b64-bef9-f738fb7e12fc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:40.546696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 666/1354] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:40.552427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e03ec-82ec-4805-89f0-b6fc28cf43c6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:40.552879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e03ec-82ec-4805-89f0-b6fc28cf43c6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:40.553227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e03ec-82ec-4805-89f0-b6fc28cf43c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:40.632612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-367e03ec-82ec-4805-89f0-b6fc28cf43c6 None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:40.633822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-367e03ec-82ec-4805-89f0-b6fc28cf43c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:40.744741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 689/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:43.956774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119da8-e51d-41bb-9250-c5d2432400b0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:43.957392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119da8-e51d-41bb-9250-c5d2432400b0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:43.957392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119da8-e51d-41bb-9250-c5d2432400b0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:43.958800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 667/1356] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:43.964216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefec1b8-63d0-4e2c-a826-9d260103e7b1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:43.964652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefec1b8-63d0-4e2c-a826-9d260103e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:43.964914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefec1b8-63d0-4e2c-a826-9d260103e7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:44.020841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:44.056011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aefec1b8-63d0-4e2c-a826-9d260103e7b1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:44.056011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aefec1b8-63d0-4e2c-a826-9d260103e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:44.181106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 690/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:44.479443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:44.480195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:44.480774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:44.482524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 668/1358] 10.222.0.40 () {54 vars in 962 bytes} [Sun Aug 28 10:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:44.492378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c3a6a-b105-4e8c-bfd3-a7734399f2a8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:44.493020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c3a6a-b105-4e8c-bfd3-a7734399f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:44.493514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c3a6a-b105-4e8c-bfd3-a7734399f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:44.589938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-468c3a6a-b105-4e8c-bfd3-a7734399f2a8 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:44.591264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-468c3a6a-b105-4e8c-bfd3-a7734399f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:44.646627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:44.727102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 691/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Aug 28 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:44.740234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:44.742510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:44.905923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904ff5-d411-482c-ad4f-bfe526bfe90a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:44.987872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:45.030400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:45.031006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:45.031338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:45.031602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:45.033335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:45.131777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:45.133694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2fedc-6cce-4c3c-9155-ccb11b9770e3 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:45.156382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 669/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:08:47.293086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:47.293729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:47.293729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:47.296548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 692/1361] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:47.300640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215568-100d-4356-a946-96b729792c0c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:47.301438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215568-100d-4356-a946-96b729792c0c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:47.301544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215568-100d-4356-a946-96b729792c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:47.378171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-59215568-100d-4356-a946-96b729792c0c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:47.379650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-59215568-100d-4356-a946-96b729792c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:47.434402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:47.531526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 670/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:47.823887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:47.825525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:47.985090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a2e1-fbcb-474b-86e4-c1928acca102 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:47.997006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:48.133034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:48.133658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:48.133933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:48.134198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:48.135807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:48.233209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:48.234994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4113-d309-4a38-8340-7003609937e5 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:48.256955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 693/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:50.487634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e70e1-6272-48ad-a754-c3ea676214ed None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:50.488208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e70e1-6272-48ad-a754-c3ea676214ed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:50.488208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e70e1-6272-48ad-a754-c3ea676214ed None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:50.489802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 671/1364] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:50.494958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30ba3-6bcb-4ada-a2ed-5b162623b93c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:50.495417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30ba3-6bcb-4ada-a2ed-5b162623b93c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:50.495516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30ba3-6bcb-4ada-a2ed-5b162623b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:50.512695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:50.566807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-80b30ba3-6bcb-4ada-a2ed-5b162623b93c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:50.568055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80b30ba3-6bcb-4ada-a2ed-5b162623b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:50.669666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 694/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:50.952627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:50.952912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:50.953180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:50.954528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 672/1366] 10.222.0.40 () {54 vars in 962 bytes} [Sun Aug 28 10:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:50.960417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f5b8-c4a1-4cf2-bb26-a607ba18c189 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:50.961348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f5b8-c4a1-4cf2-bb26-a607ba18c189 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:50.961488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f5b8-c4a1-4cf2-bb26-a607ba18c189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:51.042814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f3e5f5b8-c4a1-4cf2-bb26-a607ba18c189 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:51.044195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f3e5f5b8-c4a1-4cf2-bb26-a607ba18c189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:51.082026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:51.170112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 695/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Aug 28 10:08:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:51.178238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:51.179747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:51.333196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b05640-9a24-4512-81a0-8c8578787c76 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:51.354576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:51.453222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6020418-7544-4945-804d-06477af65436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:51.453844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6020418-7544-4945-804d-06477af65436 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:51.454142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6020418-7544-4945-804d-06477af65436 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:51.454407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6020418-7544-4945-804d-06477af65436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:51.456292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f6020418-7544-4945-804d-06477af65436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:51.554202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6020418-7544-4945-804d-06477af65436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:51.556042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6020418-7544-4945-804d-06477af65436 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:51.575336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 673/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:52.086731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0309-78a6-4fb8-8d86-bcdec7ac34fc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:52.087305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0309-78a6-4fb8-8d86-bcdec7ac34fc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:52.087305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0309-78a6-4fb8-8d86-bcdec7ac34fc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:52.088794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 696/1369] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:52.094060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822b58-cd8b-4c05-87ce-d2d746d2ede9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:52.094350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822b58-cd8b-4c05-87ce-d2d746d2ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:52.094623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822b58-cd8b-4c05-87ce-d2d746d2ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:52.128293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:52.165575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a7822b58-cd8b-4c05-87ce-d2d746d2ede9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:52.166799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7822b58-cd8b-4c05-87ce-d2d746d2ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:52.272577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 674/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:52.820383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:52.820815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:52.821187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:52.822845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 697/1371] 10.222.0.40 () {54 vars in 964 bytes} [Sun Aug 28 10:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:52.828579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc346-89e7-4f5d-8836-21417d29f5b0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:52.829028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc346-89e7-4f5d-8836-21417d29f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:52.829199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc346-89e7-4f5d-8836-21417d29f5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:52.909558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-032dc346-89e7-4f5d-8836-21417d29f5b0 None None] MFA Rules not processed for user `77f82ddcd0604a3697dc8b571719e1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:52.911055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-032dc346-89e7-4f5d-8836-21417d29f5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:52.960149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:52.993269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 675/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Aug 28 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:53.001093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:53.002643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:53.112324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b62a-2a4e-49f7-a457-d22fc3783d29 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:53.157186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:53.197866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:53.198661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:53.199057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:53.199444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:53.201419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:53.325775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:53.327240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d9c3c-f541-47fb-a828-1a01b473f705 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:53.346418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 698/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:53.554320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:53.554604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:53.554876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:53.556351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 676/1374] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:53.561661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4106-70cd-4de4-ae85-5ed5d79e85bc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:53.561969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4106-70cd-4de4-ae85-5ed5d79e85bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:53.562242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4106-70cd-4de4-ae85-5ed5d79e85bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:53.633282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0fca4106-70cd-4de4-ae85-5ed5d79e85bc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:53.634468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fca4106-70cd-4de4-ae85-5ed5d79e85bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:53.656100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:53.733536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 699/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:54.018747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:54.019845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:54.170975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c587ab-874b-4bbe-bb6c-0683c7b93814 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:54.232140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:54.293889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:54.294423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:54.294696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:54.294958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:54.296554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:54.390407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:54.391992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911eb81-8f56-4908-a67e-df10b13fd946 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:54.410046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 677/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:56.346898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:56.347399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:56.347503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:56.348957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 700/1377] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:08:56.354078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aab6f4-71ee-422b-8921-3ce1cf09429e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:56.354521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aab6f4-71ee-422b-8921-3ce1cf09429e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:56.354617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aab6f4-71ee-422b-8921-3ce1cf09429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:56.435299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-53aab6f4-71ee-422b-8921-3ce1cf09429e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:56.437012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53aab6f4-71ee-422b-8921-3ce1cf09429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:56.444288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:56.536363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 678/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:56.825334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:56.826299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:57.000289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71c00-9bf9-48b6-9586-279e41bbb411 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:57.004658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:57.127544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:57.128319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:57.128735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:57.129102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:57.130947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:57.225310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:08:57.227078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3443368-61f5-4fd0-9e3c-9c6f16e43069 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:08:57.245612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 701/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:08:59.181481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519df7f-b537-4644-989c-197ab3856591 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:59.182143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519df7f-b537-4644-989c-197ab3856591 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:59.182143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519df7f-b537-4644-989c-197ab3856591 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:59.184704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 679/1380] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 28 10:08:59.188981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5549fa-200f-47dc-a5ae-6724e9c67e2e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:59.189453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5549fa-200f-47dc-a5ae-6724e9c67e2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:59.189536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5549fa-200f-47dc-a5ae-6724e9c67e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:59.201473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:59.265667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5b5549fa-200f-47dc-a5ae-6724e9c67e2e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:08:59.267255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b5549fa-200f-47dc-a5ae-6724e9c67e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:59.382059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 702/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:08:59.670800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519df7f-b537-4644-989c-197ab3856591 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:08:59.672742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3519df7f-b537-4644-989c-197ab3856591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:08:59.821963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:08:59.834648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519df7f-b537-4644-989c-197ab3856591 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:08:59.957982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:08:59.958673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:08:59.959020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:08:59.959355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:08:59.961040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:00.059344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:00.061291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960836b-0643-4e3d-bfef-7c957d402418 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:00.072646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:00.082513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 680/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:02.326690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:02.327228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:02.327343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:02.329060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 703/1383] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:02.336109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ff510-0cd0-478c-be1a-b69b891f59cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:02.336613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ff510-0cd0-478c-be1a-b69b891f59cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:02.337007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ff510-0cd0-478c-be1a-b69b891f59cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:02.438083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ff9ff510-0cd0-478c-be1a-b69b891f59cf None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:02.439962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ff9ff510-0cd0-478c-be1a-b69b891f59cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:02.565701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 681/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:02.918269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:02.920234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:03.024803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:03.097340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b112c64-728a-43b9-8db5-6f84754529cf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:03.253707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:03.254519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:03.255010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:03.255513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:03.257932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:03.327677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:03.375752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:03.378137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5ed49-4b78-4590-b3ec-c7b82c738e0a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:03.398850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 704/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:05.509339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:05.510091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:05.510091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:05.511489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 682/1386] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:05.517486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060473-3705-409f-a043-7e6266302098 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:05.518079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060473-3705-409f-a043-7e6266302098 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:05.518079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060473-3705-409f-a043-7e6266302098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:05.594362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e3060473-3705-409f-a043-7e6266302098 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:05.595841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3060473-3705-409f-a043-7e6266302098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:05.705342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 705/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:06.003836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:06.005355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:06.098568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:06.188130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153ed8b-e991-46bf-ac19-c736057d08f5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:06.314323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:06.314884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:06.315173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:06.315444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:06.317182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:06.358039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:06.412529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:06.414129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18a9ad-7fb8-4e6b-be63-f13b050c48e0 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:06.433494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 683/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:08.252079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b535f2b-c90f-428e-8c8b-caac048d9593 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.252791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b535f2b-c90f-428e-8c8b-caac048d9593 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.252791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b535f2b-c90f-428e-8c8b-caac048d9593 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.254010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 706/1389] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:08.259566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc36c-3ba7-4e19-9356-230af4a0a607 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.260034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc36c-3ba7-4e19-9356-230af4a0a607 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.260116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abc36c-3ba7-4e19-9356-230af4a0a607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.329246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-04abc36c-3ba7-4e19-9356-230af4a0a607 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:08.330358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-04abc36c-3ba7-4e19-9356-230af4a0a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:08.351631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91925-953d-4295-9bc9-9d1c2de00b5a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.351891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91925-953d-4295-9bc9-9d1c2de00b5a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.352171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91925-953d-4295-9bc9-9d1c2de00b5a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.353820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 707/1390] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:08.359199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf253-b0b5-4394-af60-0205b2418c1b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.359502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf253-b0b5-4394-af60-0205b2418c1b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.359771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf253-b0b5-4394-af60-0205b2418c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.397146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:08.429424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-22bdf253-b0b5-4394-af60-0205b2418c1b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:08.430654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-22bdf253-b0b5-4394-af60-0205b2418c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:08.438892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 684/1391] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:08.531249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 708/1392] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:08.741860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.742232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.742540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.743918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 685/1393] 10.222.0.40 () {54 vars in 967 bytes} [Sun Aug 28 10:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:08.749298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:08.749582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:08.749852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:08.816548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:08.818143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:08.832042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:08.834353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] MFA Rules not processed for user `32fb55fa700442dd974594ada94e7bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:08.835568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:08.883385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:08.911892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 709/1394] 10.222.0.40 () {58 vars in 1064 bytes} [Sun Aug 28 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:08.919293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:08.927447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e82894-1d51-4a9f-abce-5903750d4459 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:08.964339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027afc3-aef2-4e15-b593-e56d28ae753d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:09.001232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:09.001841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:09.002205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:09.002387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:09.003994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:09.049786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:09.077166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:09.077787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:09.078062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:09.078327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:09.079928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:09.113597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:09.115072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67a2ca-7a68-41b6-81a5-fb34ce757e2e service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:09.129226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:09.135433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 710/1395] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:09.168032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:09.169664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e9491-1071-4aa1-ba8e-189f66533774 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:09.188261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 686/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:09:09.883033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4dee-1c5b-406e-bbd6-1dcd8148ea68 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:09.883580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4dee-1c5b-406e-bbd6-1dcd8148ea68 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:09.883809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4dee-1c5b-406e-bbd6-1dcd8148ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:09.951867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc0d4dee-1c5b-406e-bbd6-1dcd8148ea68 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:09.953117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc0d4dee-1c5b-406e-bbd6-1dcd8148ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:10.041768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 711/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:09:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:10.089278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:10.089801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:10.089801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:10.091201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 687/1398] 10.222.0.40 () {54 vars in 943 bytes} [Sun Aug 28 10:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:10.097723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cbd91-0710-4a7b-8340-f4aaadacb518 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:10.098039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cbd91-0710-4a7b-8340-f4aaadacb518 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:10.098339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cbd91-0710-4a7b-8340-f4aaadacb518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:10.179037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-107cbd91-0710-4a7b-8340-f4aaadacb518 None None] MFA Rules not processed for user `7c9ccc207322482e818b7c9f49aeed29`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:10.180240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-107cbd91-0710-4a7b-8340-f4aaadacb518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:10.211340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:10.293924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 712/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Sun Aug 28 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:10.304290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:10.305908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:10.408083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:10.451001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3afef-6c8a-4df0-b7a9-3a0cded8850b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:10.556060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:10.556712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:10.556989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:10.557278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:10.558870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:10.646967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:10.648633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634f170-4fb1-455e-a030-b8cf96658638 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:10.652126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:10.658046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 688/1400] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:09:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:09:11.136892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8865-77e9-422f-9222-881462b290f3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:11.137460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8865-77e9-422f-9222-881462b290f3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:11.137644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8865-77e9-422f-9222-881462b290f3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:11.139109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 713/1401] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:11.144649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe44aa-0225-4a9c-9188-a71b5b829b09 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:11.145152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe44aa-0225-4a9c-9188-a71b5b829b09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:11.145398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe44aa-0225-4a9c-9188-a71b5b829b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:11.217852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4dfe44aa-0225-4a9c-9188-a71b5b829b09 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:11.219069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4dfe44aa-0225-4a9c-9188-a71b5b829b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:11.324599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 689/1402] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:09:11.613227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8865-77e9-422f-9222-881462b290f3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:11.614809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-780d8865-77e9-422f-9222-881462b290f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:11.784970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8865-77e9-422f-9222-881462b290f3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:11.788294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:11.921061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:11.921669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:11.921938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:11.922203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:11.923801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:12.022600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:12.024316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe3097-8d17-43f1-913e-0263dc6c399f service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:12.044478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 714/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:14.010533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:14.011145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:14.011145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:14.012597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 690/1404] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:14.017667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716249-d666-4180-ba47-0003bb577d1e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:14.018095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716249-d666-4180-ba47-0003bb577d1e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:14.018245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716249-d666-4180-ba47-0003bb577d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:14.028036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:14.098338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8c716249-d666-4180-ba47-0003bb577d1e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:14.101632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c716249-d666-4180-ba47-0003bb577d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:14.202170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 715/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:14.493008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:14.494547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:14.588039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:14.664685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86ba70-0b37-4a19-9fdf-adb3f0e2d91a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:14.798042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:14.798999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:14.799595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:14.800040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:14.802258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:14.846355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:14.907365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:14.908995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b798d-6917-4b86-9e5d-4063cbfc4f4b service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:14.929958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 691/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:16.857639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:16.858172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:16.858172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:16.862570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 716/1407] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:16.864899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeab87-a47b-403d-ab91-cd96f9f00557 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:16.865349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeab87-a47b-403d-ab91-cd96f9f00557 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:16.865444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeab87-a47b-403d-ab91-cd96f9f00557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:16.936480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6cdeab87-a47b-403d-ab91-cd96f9f00557 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:16.937835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6cdeab87-a47b-403d-ab91-cd96f9f00557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:17.038191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 692/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:17.325909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:17.327718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:17.427785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:17.501240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738f089-b56e-4b6b-a649-8e3672b80e07 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:17.632704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:17.633451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:17.633793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:17.634156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:17.635765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:17.688844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:17.736297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:17.737964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626048a-c61e-4c5a-b4a1-0c8a516a3e9f service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:17.761892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 717/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:27.036396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:27.759491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=24 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:32.613480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c999db-df47-423a-ad76-82e9acdbbc5c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:32.613936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c999db-df47-423a-ad76-82e9acdbbc5c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:32.614030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c999db-df47-423a-ad76-82e9acdbbc5c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:32.615443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 693/1410] 10.222.0.40 () {54 vars in 918 bytes} [Sun Aug 28 10:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:32.621005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa6a8-abf7-4d15-b4fc-ecae41dbbc3a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:32.621415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa6a8-abf7-4d15-b4fc-ecae41dbbc3a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:32.621535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfa6a8-abf7-4d15-b4fc-ecae41dbbc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:32.707024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dbcfa6a8-abf7-4d15-b4fc-ecae41dbbc3a None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:32.708262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbcfa6a8-abf7-4d15-b4fc-ecae41dbbc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:32.817103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 718/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Aug 28 10:09:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:32.833038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305317-e7be-4303-bbee-282ee0858e23 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:32.833384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305317-e7be-4303-bbee-282ee0858e23 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:32.833718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305317-e7be-4303-bbee-282ee0858e23 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:32.835741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 694/1412] 10.222.0.40 () {54 vars in 918 bytes} [Sun Aug 28 10:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:32.842458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd865-3eab-482c-8d66-a8c2023265ad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:32.842653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd865-3eab-482c-8d66-a8c2023265ad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:32.842943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd865-3eab-482c-8d66-a8c2023265ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:32.919785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:32.922946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fdfbd865-3eab-482c-8d66-a8c2023265ad None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:32.924521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fdfbd865-3eab-482c-8d66-a8c2023265ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:33.046944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 719/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Aug 28 10:09:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:09:35.477770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:35.478365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:35.478365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:35.479656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:35.486069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f51e-3c21-4c6e-ac1b-4f7d08d219ea None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:35.486550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f51e-3c21-4c6e-ac1b-4f7d08d219ea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:35.486646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525f51e-3c21-4c6e-ac1b-4f7d08d219ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:35.563705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3525f51e-3c21-4c6e-ac1b-4f7d08d219ea None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:35.564865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3525f51e-3c21-4c6e-ac1b-4f7d08d219ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:35.624388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:35.666818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:35.958887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:35.961332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:36.136770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e56a-415d-4859-9e65-5eb39dbf00a5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:36.210922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:36.252533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:36.253153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:36.253423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:36.253689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:36.255276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:36.346388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:36.348021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e7036-3bdc-4a11-b580-1b08be8361e0 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:36.366178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 696/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:37.253256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:37.253881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:37.253881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:37.255151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:37.261494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0984aa-3919-4f51-ae80-bd46c110d8a0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:37.261955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0984aa-3919-4f51-ae80-bd46c110d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:37.262060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0984aa-3919-4f51-ae80-bd46c110d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:37.359015 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1f0984aa-3919-4f51-ae80-bd46c110d8a0 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:37.360242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f0984aa-3919-4f51-ae80-bd46c110d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:37.388941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:37.466438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:37.623503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:37.625118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:37.792973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2840-cfe5-48be-aa70-d92f8acba5f2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:37.838241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:37.914756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:37.915363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:37.915627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:37.915903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:37.917541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:38.015664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:38.017397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d301e8d-af11-48b5-9586-f5c936095931 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:38.035867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 722/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:38.917550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:38.918116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:38.918116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:38.919482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:38.926115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020094c-7abc-4ad9-89a2-8d0a05d9ecef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:38.926814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020094c-7abc-4ad9-89a2-8d0a05d9ecef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:38.926963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020094c-7abc-4ad9-89a2-8d0a05d9ecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:38.990408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:39.012759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e020094c-7abc-4ad9-89a2-8d0a05d9ecef None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:39.013957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e020094c-7abc-4ad9-89a2-8d0a05d9ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:39.111461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:39.277629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:39.280039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:39.449985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09b43b-5554-4c7d-b077-0f6c6f2f1920 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:39.468221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:39.578311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:39.578829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:39.579223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:39.579498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:39.581213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:39.677766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:39.679382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ead77c-eeed-43d9-88f0-90b601a2494f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:39.698651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 699/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:40.569178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:40.569772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:40.569772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:40.571106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:40.577330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ffc34-97b8-481f-815d-08ad652cdfeb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:40.577916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ffc34-97b8-481f-815d-08ad652cdfeb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:40.578025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ffc34-97b8-481f-815d-08ad652cdfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:40.599649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:40.656788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c08ffc34-97b8-481f-815d-08ad652cdfeb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:40.658006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c08ffc34-97b8-481f-815d-08ad652cdfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:40.758889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:40.909613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:40.911596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:41.043014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:41.088910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369501ff-eaa2-413c-9af9-67fedf4f65bb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:41.219888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:41.220530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:41.220832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:41.221094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:41.223429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:41.309505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:41.330607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:41.332768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecbafe-0e94-4e49-9183-948bb5a4b639 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:41.351363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 725/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:42.218599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd6e21-794e-412f-b959-6be504100139 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:42.219160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd6e21-794e-412f-b959-6be504100139 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:42.219160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd6e21-794e-412f-b959-6be504100139 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:42.220536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:42.226727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e899-4264-40ea-82b4-c9c5b3144f6c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:42.227455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e899-4264-40ea-82b4-c9c5b3144f6c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:42.227644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e899-4264-40ea-82b4-c9c5b3144f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:42.309437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5440e899-4264-40ea-82b4-c9c5b3144f6c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:42.310545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5440e899-4264-40ea-82b4-c9c5b3144f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:42.424030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:42.579207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd6e21-794e-412f-b959-6be504100139 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:42.580810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-33bd6e21-794e-412f-b959-6be504100139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:42.676444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:42.743315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd6e21-794e-412f-b959-6be504100139 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:42.867647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:42.868259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:42.868571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:42.868823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:42.870412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:42.921358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:42.965906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:42.967490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a12c3-dbd1-407e-9146-9c2ea448a26f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:42.987070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 702/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:43.786699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:43.787267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:43.787267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:43.788605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:43.794868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f3e3-b0a4-4c70-8209-16f003b2d4b9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:43.795600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f3e3-b0a4-4c70-8209-16f003b2d4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:43.795855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527f3e3-b0a4-4c70-8209-16f003b2d4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:43.880913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1527f3e3-b0a4-4c70-8209-16f003b2d4b9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:43.882150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1527f3e3-b0a4-4c70-8209-16f003b2d4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:43.980652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:44.132626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:44.134119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:44.184195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:44.285362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16091c36-93a7-4cf3-8ac1-8b8cc72314c3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:44.403334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:44.403868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:44.404141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:44.404405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:44.406040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:44.413201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:44.504337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:44.505980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b327-0e53-436c-bce6-36f2f45e833c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:44.524235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 728/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:45.335818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:45.336746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:45.336901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:45.339098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:45.344192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36363-0734-4e0a-89f1-a235d273cde3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:45.344659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36363-0734-4e0a-89f1-a235d273cde3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:45.344766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c36363-0734-4e0a-89f1-a235d273cde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:45.423205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f4c36363-0734-4e0a-89f1-a235d273cde3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:45.424314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4c36363-0734-4e0a-89f1-a235d273cde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:45.497061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:45.527626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:45.681653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:45.684059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:45.705659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:45.842989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4177-19ea-4781-8619-fd1ab2a815c8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:45.952955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:45.981359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:45.981975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:45.982252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:45.982515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:45.984123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:46.077786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:46.079409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f36db-c339-4c2f-ac36-6101d20907aa service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:46.097908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 705/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:46.898110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:46.898654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:46.898654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:46.900297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:46.905599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033139d-b803-489b-bf39-34cd21788ad9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:46.906049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033139d-b803-489b-bf39-34cd21788ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:46.906145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033139d-b803-489b-bf39-34cd21788ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:46.995202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e033139d-b803-489b-bf39-34cd21788ad9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:46.996414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e033139d-b803-489b-bf39-34cd21788ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:47.038213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:47.099198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:47.251745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:47.253209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:47.408775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d1bc-13fa-43f8-983a-69d09cfbf802 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:47.463760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:47.527524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:47.528050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:47.528325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:47.528620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:47.530223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:47.623137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:47.624769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec5a95-d042-442e-9378-c81dda2f2edf service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:47.643116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 731/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:48.434028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:48.434490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:48.434596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:48.435960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:48.441835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7d7ec-ad23-457d-a333-ae9f407ac3f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:48.442295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7d7ec-ad23-457d-a333-ae9f407ac3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:48.442408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7d7ec-ad23-457d-a333-ae9f407ac3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:48.522704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a3d7d7ec-ad23-457d-a333-ae9f407ac3f8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:48.524048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a3d7d7ec-ad23-457d-a333-ae9f407ac3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:48.526150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:48.629474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:48.778892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:48.781307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:48.945305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49645f-cf95-4511-ae21-9c5bf8e214bf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:48.974083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:49.075141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:49.075674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:49.075948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:49.076216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:49.077855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:49.170563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:49.172147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7af607-bc41-4d8c-840f-c3ebca1e2986 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:49.190711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 708/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:51.010903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:51.011835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:51.011835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:51.013993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:51.021987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22bfc4-a291-4bca-ac04-08951c47770e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:51.022402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22bfc4-a291-4bca-ac04-08951c47770e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:51.022514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22bfc4-a291-4bca-ac04-08951c47770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:51.072273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:51.096842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9b22bfc4-a291-4bca-ac04-08951c47770e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:51.098084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9b22bfc4-a291-4bca-ac04-08951c47770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:51.199826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:51.484419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:51.486841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:51.602273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a959b5c-121f-4211-8ba8-ce69e799274e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:51.634197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:51.679864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:51.680448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:51.680751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:51.681015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:51.682615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:51.798142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:51.799833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4db4-0176-4714-8376-f8f89b3cd012 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:51.818198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 734/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:53.862990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:53.863555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:53.863555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:53.865730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:09:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:53.874547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d308c-aa55-49e2-9159-de7abd9d5c64 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:53.874959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d308c-aa55-49e2-9159-de7abd9d5c64 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:53.875125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d308c-aa55-49e2-9159-de7abd9d5c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:53.948026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dd1d308c-aa55-49e2-9159-de7abd9d5c64 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:53.949260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd1d308c-aa55-49e2-9159-de7abd9d5c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:53.962650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:54.050421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:54.335922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:54.337495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:54.459818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99534cb5-22be-48b4-b301-6734263cca71 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:54.473867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:54.540498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:54.541558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:54.542032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:54.542492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:54.545377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:54.665870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:54.667328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfd385-d12a-48ba-8ef4-62ebfb8511e5 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:54.686248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 711/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:56.985630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:56.986534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:56.986534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:56.988753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:09:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:56.994179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f7d8-d4ca-4c4a-9cb9-4900dc0cff14 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:56.994631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f7d8-d4ca-4c4a-9cb9-4900dc0cff14 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:56.994739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f7d8-d4ca-4c4a-9cb9-4900dc0cff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:57.043832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:57.063052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5926f7d8-d4ca-4c4a-9cb9-4900dc0cff14 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:57.064260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5926f7d8-d4ca-4c4a-9cb9-4900dc0cff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:57.163532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:09:57.443689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:09:57.445299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:57.563161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac572d-f7ac-47a0-ac7c-c68716235892 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:09:57.606934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:57.643626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:09:57.644227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:57.644494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:57.644789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:57.646381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:57.760305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:09:57.761724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0231fd1-ae7e-4d2d-9a96-fb4aacba92b5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:09:57.782228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 737/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:09:59.796182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:59.797093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:59.797093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:59.799581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:09:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:09:59.805231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4519d-7039-4256-a375-2151dd35c683 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:09:59.805676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4519d-7039-4256-a375-2151dd35c683 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:09:59.805771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4519d-7039-4256-a375-2151dd35c683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:09:59.875883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7ff4519d-7039-4256-a375-2151dd35c683 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:09:59.877150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7ff4519d-7039-4256-a375-2151dd35c683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:09:59.899373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:09:59.986877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:00.268943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:00.270634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:00.384648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4755e4-06b8-4ff7-8187-1bd269dd4738 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:00.404892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:00.458890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:00.459502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:00.459779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:00.460052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:00.461707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:00.577061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:00.581453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3636bc1-1106-4a1b-a24b-1b34bf393f73 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:00.599079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 714/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:02.550312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:02.551024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:02.551024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:02.552400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:02.558166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ed7e-98fc-4cb4-9573-e8bf1695dbb8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:02.558623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ed7e-98fc-4cb4-9573-e8bf1695dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:02.558735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ed7e-98fc-4cb4-9573-e8bf1695dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:02.627469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:02.631140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ec39ed7e-98fc-4cb4-9573-e8bf1695dbb8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:02.632983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec39ed7e-98fc-4cb4-9573-e8bf1695dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:02.734815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:03.018529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:03.021235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:03.146467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501fff4-34a8-4683-b652-806fe3363d1d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:03.196187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:03.224577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:03.225210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:03.225469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:03.225738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:03.227337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:03.352285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:03.353719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97cd4c-dfb1-4aa4-9b08-7cbee7e20c8a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:03.372793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 740/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:05.318785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:05.319717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:05.319717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:05.321870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:05.328308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f914b78-11f6-4acc-8572-bd4fcc6124c9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:05.328799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f914b78-11f6-4acc-8572-bd4fcc6124c9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:05.328905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f914b78-11f6-4acc-8572-bd4fcc6124c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:05.401602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2f914b78-11f6-4acc-8572-bd4fcc6124c9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:05.402825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2f914b78-11f6-4acc-8572-bd4fcc6124c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:05.433608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:05.513114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:05.799915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:05.802539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:05.925545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e72aa9-3815-4165-91e7-453fe2b90cfe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:05.956909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:06.019168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:06.019809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:06.020094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:06.020360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:06.021998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:06.139147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:06.140785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae040620-3e25-4ca1-be8d-448b88359fc6 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:06.161749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 717/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:08.137801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:08.138644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:08.138644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:08.140875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:08.150105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53a4a4-e800-47ab-9554-435fe0f95238 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:08.150548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53a4a4-e800-47ab-9554-435fe0f95238 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:08.150760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53a4a4-e800-47ab-9554-435fe0f95238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:08.225333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-db53a4a4-e800-47ab-9554-435fe0f95238 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:08.226552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-db53a4a4-e800-47ab-9554-435fe0f95238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:08.235029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:08.340922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:08.627412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:08.629052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:08.744493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d45ff-0ef5-4445-ad5d-2dc244928983 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:08.802518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:08.823767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:08.824702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:08.824977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:08.825239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:08.826865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:08.941103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:08.942473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc574309-1a0a-48dc-8a08-58f568274da8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:08.961802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 743/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:10.917195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:10.918077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:10.918077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:10.920347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:10.927136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9c85a-f3e5-4ecf-804c-f048da77ceef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:10.927582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9c85a-f3e5-4ecf-804c-f048da77ceef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:10.927704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9c85a-f3e5-4ecf-804c-f048da77ceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:11.000780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-10c9c85a-f3e5-4ecf-804c-f048da77ceef None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:11.002008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10c9c85a-f3e5-4ecf-804c-f048da77ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:11.041011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:11.109072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:11.394974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:11.396562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:11.517534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8068cf-6c97-40ff-aefb-a03684c82b55 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:11.561579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:11.603177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:11.603794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:11.604072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:11.604342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:11.605988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:11.727564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:11.728970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287730b-45d4-4d60-80db-2591b89a257b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:11.748161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 720/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:13.664484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:13.665070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:13.665200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:13.666509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:13.675081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da036e6-4ed9-405b-b89c-4400ee71a4c8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:13.675502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da036e6-4ed9-405b-b89c-4400ee71a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:13.675659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da036e6-4ed9-405b-b89c-4400ee71a4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:13.744972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4da036e6-4ed9-405b-b89c-4400ee71a4c8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:13.746237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4da036e6-4ed9-405b-b89c-4400ee71a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:13.765654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:13.848559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:14.124816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:14.126379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:14.233931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101c8f4-2f43-4448-9fad-e216827b366f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:14.305560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:14.312820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:14.313433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:14.313711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:14.313980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:14.315578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:14.444024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:14.445425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d5fd1-b537-41f7-9606-df414a20e4ef service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:14.464157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 746/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:16.389330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:16.389961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:16.389961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:16.391256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:16.396749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7872e-676d-4765-9790-9f2f7a98a4be None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:16.397213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7872e-676d-4765-9790-9f2f7a98a4be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:16.397315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7872e-676d-4765-9790-9f2f7a98a4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:16.470846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-72c7872e-676d-4765-9790-9f2f7a98a4be None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:16.472105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-72c7872e-676d-4765-9790-9f2f7a98a4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:16.522187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:16.582106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:16.859991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:16.861582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:16.977496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc042-2e2c-4329-a2c2-21935c4e2bc8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:17.026084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:17.053308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:17.053908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:17.054188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:17.054458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:17.056055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:17.171645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:17.173492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf83f0-e3dc-4e01-80f5-2cde1723c120 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:17.193995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 723/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:10:26.580148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=22 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:27.191245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=66 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:36.604017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b0752-3f27-4545-a50a-2b3b4162635e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:36.604563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b0752-3f27-4545-a50a-2b3b4162635e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:36.604563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b0752-3f27-4545-a50a-2b3b4162635e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:36.605978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 748/1471] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:36.611719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05f0c-48f5-4451-9508-035db9f29099 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:36.612162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05f0c-48f5-4451-9508-035db9f29099 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:36.612259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05f0c-48f5-4451-9508-035db9f29099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:36.683869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fae05f0c-48f5-4451-9508-035db9f29099 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:36.685120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fae05f0c-48f5-4451-9508-035db9f29099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:36.784126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 724/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:37.085072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:37.085359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:37.085639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:37.086988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 749/1473] 10.222.0.40 () {54 vars in 967 bytes} [Sun Aug 28 10:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:37.091848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f12cf-03a2-46cd-9a56-63c77d84c9ac None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:37.092131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f12cf-03a2-46cd-9a56-63c77d84c9ac None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:37.092401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f12cf-03a2-46cd-9a56-63c77d84c9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:37.169627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-498f12cf-03a2-46cd-9a56-63c77d84c9ac None None] MFA Rules not processed for user `32fb55fa700442dd974594ada94e7bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:37.170825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-498f12cf-03a2-46cd-9a56-63c77d84c9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:37.193414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:37.258242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 725/1474] 10.222.0.40 () {58 vars in 1064 bytes} [Sun Aug 28 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:37.266245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:37.267801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:37.379302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8868e1-c916-4931-8597-2b157caaaa31 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:37.459113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:37.459724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:37.459999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:37.460266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:37.461933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:37.518655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:37.574531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:37.575986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e4ffa-bbfd-46a0-a4d0-ca458c33c31f service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:37.599514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 750/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:10:39.632607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:39.633110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:39.633110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:39.634779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 726/1476] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:39.641464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e6f9-4b4d-453b-b492-3581dbf26c35 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:39.641904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e6f9-4b4d-453b-b492-3581dbf26c35 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:39.642024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9e6f9-4b4d-453b-b492-3581dbf26c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:39.717766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0fa9e6f9-4b4d-453b-b492-3581dbf26c35 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:39.719002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fa9e6f9-4b4d-453b-b492-3581dbf26c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:39.830264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 751/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:40.107258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:40.109692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:40.244668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7bffa-98ac-44fb-b4bb-5e6164b5992a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:40.295835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:40.343739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:40.344281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:40.344662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:40.344933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:40.345577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:40.347304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:10:40.347304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:10:40.347304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:10:40.348432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81beed-c12b-4994-a84e-2c985b382dcb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:40.369140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 727/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Aug 28 10:10:40 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:10:42.280274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:42.280885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:42.280885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:42.282286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 752/1479] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:42.288712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc2fd-4959-448e-a651-fe61dd1b6f45 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:42.289168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc2fd-4959-448e-a651-fe61dd1b6f45 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:42.289298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cc2fd-4959-448e-a651-fe61dd1b6f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:42.362754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8c7cc2fd-4959-448e-a651-fe61dd1b6f45 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:42.364032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8c7cc2fd-4959-448e-a651-fe61dd1b6f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:42.469810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 728/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:42.768853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:42.770420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:42.780352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:42.936987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f448e7e-2960-488b-a572-fa8490378425 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:43.024142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:43.058237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:43.058853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:43.059124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:43.059389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:43.061059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:43.153112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:43.154769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1158-0367-48d7-be2b-07346d26d8e3 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:43.174345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 753/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:45.286308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:45.286896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:45.286896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:45.288944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 729/1482] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:45.297555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a7f1-5c35-4a65-a44f-72b47d3e83ae None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:45.298026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a7f1-5c35-4a65-a44f-72b47d3e83ae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:45.298135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a7f1-5c35-4a65-a44f-72b47d3e83ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:45.369428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2cb3a7f1-5c35-4a65-a44f-72b47d3e83ae None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:45.370686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2cb3a7f1-5c35-4a65-a44f-72b47d3e83ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:45.419666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:45.473017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 754/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:10:45.763838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:45.766364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:45.791548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:45.886297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b685-a2da-423c-9321-f1e202b21e83 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:45.962667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:45.963288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:45.963584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:45.963848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:45.965485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:46.039732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:46.081230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:46.082670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252d61-6cfa-4adb-aadc-24b23b743780 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:46.101578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 730/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:48.066359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:48.067169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:48.067169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:48.069212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 755/1485] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:48.076483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b34c3-75db-4771-8d05-fcfc05a4f86a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:48.077238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b34c3-75db-4771-8d05-fcfc05a4f86a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:48.077566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b34c3-75db-4771-8d05-fcfc05a4f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:48.149745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d50b34c3-75db-4771-8d05-fcfc05a4f86a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:48.151595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d50b34c3-75db-4771-8d05-fcfc05a4f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:48.262858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 731/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:48.554342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:48.555895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:48.680984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f444f-99a1-4481-bdf5-0dab73eb2133 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:48.755099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:48.757501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:48.758187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:48.758460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:48.758759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:48.760898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:48.873518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:48.875003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816f94a-3f5e-459c-9884-aeb12c352933 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:48.893396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 756/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:50.949720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:50.950303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:50.950303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:50.951911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 732/1488] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:50.958541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1fab-09f0-417f-9a26-6da60e980c2c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:50.959265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1fab-09f0-417f-9a26-6da60e980c2c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:50.959478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1fab-09f0-417f-9a26-6da60e980c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:51.034878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c03e1fab-09f0-417f-9a26-6da60e980c2c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:51.036102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c03e1fab-09f0-417f-9a26-6da60e980c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:51.092434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:51.138355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 757/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:51.432043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:51.434333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:51.474346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:51.548567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edf9d-5f49-4081-8256-cbadc908a02d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:51.624455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:51.625101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:51.625379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:51.625650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:51.627339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:51.718456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:51.751966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:51.753438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57836f-6824-4c9e-90e8-9c05581684ae service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:51.773307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 733/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:10:53.860634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:53.861246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:53.861246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:53.862588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 758/1491] 10.222.0.40 () {54 vars in 927 bytes} [Sun Aug 28 10:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:10:53.870090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b53e5-54b3-494f-ae9c-c781df5eb009 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:53.870876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b53e5-54b3-494f-ae9c-c781df5eb009 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:53.871058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b53e5-54b3-494f-ae9c-c781df5eb009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:53.952267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-085b53e5-54b3-494f-ae9c-c781df5eb009 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:10:53.953534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-085b53e5-54b3-494f-ae9c-c781df5eb009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:54.057087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 734/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Sun Aug 28 10:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:10:54.361193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:10:54.363662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:54.477314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163029b-247a-4c2c-b708-1ddee84833fb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:10:54.562589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:10:54.563198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:10:54.563474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:10:54.563740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:10:54.565421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:10:54.574880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:10:54.684317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:10:54.685824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9b6fe-ed1b-44f7-a6c4-684ed7e72651 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:10:54.705905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 759/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:11:04.054937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:11:04.703000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=50 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:14.561930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:14.562493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:14.562493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:14.649943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:12:14.651178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:14.759266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 735/1494] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:12:14.769410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:14.769837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:14.769837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:14.853093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:12:14.854198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:14.953681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 760/1495] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:12:14.959186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:14.960468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:14.977742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa826dd5-d56e-4746-b89e-cff41af5aa9c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.047785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:15.094388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.094986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.095276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.095537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.096203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:12:15.096498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.097529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.097529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:12:15.097529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:12:15.098406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:15.110307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 736/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:12:15.118935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:15.128531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c74c2-4dc5-421a-b7ad-9920d5eb0878 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.193848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:15.229615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.230255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.230524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.230814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.231438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.233272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.233272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:12:15.233272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:12:15.234493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:15.311498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 761/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:12:15 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:12:15.316042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:15.317608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:15.353285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204af38-99f1-4d92-9160-8808f4532e82 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.461044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.461566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.461846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.462113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.462761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.463786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.463786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:12:15.463786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:12:15.465033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:15.508650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 737/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:12:15 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:12:15.512878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:15.514412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:15.578510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79fc7a-26d2-477a-9fb0-0decda853f0a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.666611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:15.677910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.678518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.678791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.679058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.679694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.680744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.680744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:12:15.680744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:12:15.681627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:15.694078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 762/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:12:15.698420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:15.700218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:15.711907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d781-a87b-4bac-aea2-513380847b79 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.737363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:15.809284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.809886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.810169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.810436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.811083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.812087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.812087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:12:15.812087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:12:15.813012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74459f3-68f2-4097-82a9-b7de7d0a79ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:15.825474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 738/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:12:15.829980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:15.831576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:15.865669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37909d-7fff-420e-8041-eea6a0f84c13 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:15.970013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:15.970847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:15.971129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:15.971394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] PATH_INFO: `/v3/projects/a0d4f0d53bbc486ca912fd4e02320f9a/users/a1a340da4a184d24a6244fe4fc4c35a2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:15.994966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d4f0d53bbc486ca912fd4e02320f9a, user_id=a1a340da4a184d24a6244fe4fc4c35a2, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:15.996007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:15.996007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:12:15.996007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:12:15.998046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df86919-a82c-4b3d-88d8-59665d847d8d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:16.025915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 763/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:12:15 2022] PUT /identity/v3/projects/a0d4f0d53bbc486ca912fd4e02320f9a/users/a1a340da4a184d24a6244fe4fc4c35a2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:12:16.038454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595917-078d-41f5-a02a-edaa7bba780f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:16.038706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595917-078d-41f5-a02a-edaa7bba780f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:16.038972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595917-078d-41f5-a02a-edaa7bba780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:16.098173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-50595917-078d-41f5-a02a-edaa7bba780f None None] MFA Rules not processed for user `a1a340da4a184d24a6244fe4fc4c35a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:12:16.099407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50595917-078d-41f5-a02a-edaa7bba780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:16.185238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 739/1502] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:12:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:12:16.198904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:16.199191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:16.199462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:16.200887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 764/1503] 10.222.0.40 () {54 vars in 964 bytes} [Sun Aug 28 10:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:12:16.205785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c869680-0406-4ef4-84ae-99d2950b1b03 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:16.206074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c869680-0406-4ef4-84ae-99d2950b1b03 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:16.206347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c869680-0406-4ef4-84ae-99d2950b1b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:16.237555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:16.285812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4c869680-0406-4ef4-84ae-99d2950b1b03 None None] MFA Rules not processed for user `77f82ddcd0604a3697dc8b571719e1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:12:16.287041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c869680-0406-4ef4-84ae-99d2950b1b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:16.365854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 740/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Aug 28 10:12:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:12:16.378168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:16.380135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:16.435857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:16.491146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799d18-9e00-43b6-a61b-f90856c6c9ab None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:16.575189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:16.575806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:16.576081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:16.576328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:16.578019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:16.683813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:16.685303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:16.688799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:16.710876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 765/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:12:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:12:16.756434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5d62d-2031-4aa7-9fb5-6475e8658912 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:16.756932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5d62d-2031-4aa7-9fb5-6475e8658912 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:16.757036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5d62d-2031-4aa7-9fb5-6475e8658912 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:16.758318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 741/1506] 10.222.0.40 () {52 vars in 830 bytes} [Sun Aug 28 10:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:12:16.762186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:16.763461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:16.871741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e151b-a099-45d8-8b21-4ee91c0521f3 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:17.098241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:17.098781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:17.099061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:17.099327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] PATH_INFO: `/v3/users/a1a340da4a184d24a6244fe4fc4c35a2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:17.111994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a340da4a184d24a6244fe4fc4c35a2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:17.113345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:17.113345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:12:17.113345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:12:17.114549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:17.124342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:17.187970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a340da4a184d24a6244fe4fc4c35a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:17.201990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a340da4a184d24a6244fe4fc4c35a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:17.225333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a340da4a184d24a6244fe4fc4c35a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:17.237966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a340da4a184d24a6244fe4fc4c35a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:17.238575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a340da4a184d24a6244fe4fc4c35a2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:17.247592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 766/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:12:16 2022] DELETE /identity/v3/users/a1a340da4a184d24a6244fe4fc4c35a2 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:12:17.252316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:17.252628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:17.252934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:17.310836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:12:17.312031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:17.388123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:17.402361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 742/1508] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:12:17.413226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:17.415036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:17.519867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5cd9d-69bb-4ac3-a795-6625c7f3c051 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:17.578018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:17.604008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:17.604666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:17.604945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:17.605210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:17.606826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:17.733613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:17.734972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412b9b-db78-485f-a17b-30a06e2d3655 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:17.755129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 767/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:12:18.230976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:12:18.232225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:12:18.243630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836482d-3f8c-400e-a0a6-c9c8a5663e49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:12:18.343351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:12:18.343957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:12:18.344231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:12:18.344525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] PATH_INFO: `/v3/projects/a0d4f0d53bbc486ca912fd4e02320f9a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:12:18.354678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d4f0d53bbc486ca912fd4e02320f9a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:12:18.355703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:12:18.355703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:12:18.355703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:12:18.356937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:12:18.418729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d4f0d53bbc486ca912fd4e02320f9a', 'request_id': 'req-fba21475-ae40-40c5-bc97-b9694e55167d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:18.425219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d4f0d53bbc486ca912fd4e02320f9a', 'request_id': 'req-fba21475-ae40-40c5-bc97-b9694e55167d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:18.438336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d4f0d53bbc486ca912fd4e02320f9a', 'request_id': 'req-fba21475-ae40-40c5-bc97-b9694e55167d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:18.475826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] The token cache is being invalidate because project a0d4f0d53bbc486ca912fd4e02320f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:12:18.476212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:18.476777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fba21475-ae40-40c5-bc97-b9694e55167d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:12:18.478208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 743/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:12:18 2022] DELETE /identity/v3/projects/a0d4f0d53bbc486ca912fd4e02320f9a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:12:27.752766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=62 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:12:28.473547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:13:47.462920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:47.463540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:47.463540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:47.464861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 768/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:13:47.469973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e486c-cc1c-4e25-b798-6075fee6de6b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:47.470433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e486c-cc1c-4e25-b798-6075fee6de6b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:47.470536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e486c-cc1c-4e25-b798-6075fee6de6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:47.557085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-167e486c-cc1c-4e25-b798-6075fee6de6b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:13:47.558321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-167e486c-cc1c-4e25-b798-6075fee6de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:13:47.674584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 744/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:13:47.829810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:13:47.832379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:13:47.960421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dbdfe-a687-4ac6-a2c7-55b6c188b427 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:13:47.966509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:13:47.967142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:47.967420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:47.967684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:47.975967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:13:47.977593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097993ba-3b3a-44af-9e13-9e1581aa4dd6 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:13:47.987219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 769/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:13:48.826854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d4e41-e591-43d2-a573-55afcda64736 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:48.827330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d4e41-e591-43d2-a573-55afcda64736 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:48.827440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d4e41-e591-43d2-a573-55afcda64736 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:48.828809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 745/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:13:48.835617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e48d-9834-4fc6-8d1e-fb6d09e9b45c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:48.836045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e48d-9834-4fc6-8d1e-fb6d09e9b45c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:48.836138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e48d-9834-4fc6-8d1e-fb6d09e9b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:48.856286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5472e48d-9834-4fc6-8d1e-fb6d09e9b45c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:13:48.857544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5472e48d-9834-4fc6-8d1e-fb6d09e9b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:13:48.866762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 770/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:13:49.019236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d4e41-e591-43d2-a573-55afcda64736 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:13:49.036320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d4e41-e591-43d2-a573-55afcda64736 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:13:49.042873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:13:49.043478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:49.043758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:49.044031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:49.052537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:13:49.054145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d724c4f-2e1f-4f27-a3b4-44cb893fba1c service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:13:49.064692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 746/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:13:49.628924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331697-462a-439d-bb78-ffab7a7e514e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:49.629162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331697-462a-439d-bb78-ffab7a7e514e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:49.629481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47331697-462a-439d-bb78-ffab7a7e514e None None] PATH_INFO: `/v3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:49.630786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 771/1517] 10.222.0.26 () {54 vars in 925 bytes} [Sun Aug 28 10:13:49 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:13:49.638440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a747cc-db15-4018-8b80-be9fb64aa44a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:49.638728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a747cc-db15-4018-8b80-be9fb64aa44a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:49.639048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a747cc-db15-4018-8b80-be9fb64aa44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:49.705297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f9a747cc-db15-4018-8b80-be9fb64aa44a None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:13:49.706502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9a747cc-db15-4018-8b80-be9fb64aa44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:13:49.724475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:13:49.790756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 747/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Aug 28 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:13:58.851351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=56 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:13:58.901382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8c9e-4fe5-4860-b124-39c8829a53cd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:58.901753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8c9e-4fe5-4860-b124-39c8829a53cd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:58.902075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8c9e-4fe5-4860-b124-39c8829a53cd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:58.903838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 772/1519] 10.222.0.40 () {54 vars in 961 bytes} [Sun Aug 28 10:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:13:58.909088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae3bf-ce9a-41c0-b277-aec03e23285a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:58.909572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae3bf-ce9a-41c0-b277-aec03e23285a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:58.909671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae3bf-ce9a-41c0-b277-aec03e23285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:58.930872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d70ae3bf-ce9a-41c0-b277-aec03e23285a None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:13:58.932110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d70ae3bf-ce9a-41c0-b277-aec03e23285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:13:58.941914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 748/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Aug 28 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:13:58.950199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8c9e-4fe5-4860-b124-39c8829a53cd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:13:58.971727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8c9e-4fe5-4860-b124-39c8829a53cd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:13:58.977602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:13:58.978198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:13:58.978472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:13:58.978742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:13:58.987339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:13:58.988964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0e258-4db8-48d6-8d21-12fa7a0e385e service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:13:58.998665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 773/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:13:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:00.295079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3c08-e338-4480-a97c-88ab3ab32fe6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:00.295651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3c08-e338-4480-a97c-88ab3ab32fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:00.295651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3c08-e338-4480-a97c-88ab3ab32fe6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:00.297073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 749/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:00.302923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f64f-0489-4fe8-a072-b8ad585850c5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:00.303657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f64f-0489-4fe8-a072-b8ad585850c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:00.303819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f64f-0489-4fe8-a072-b8ad585850c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:00.327295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5c30f64f-0489-4fe8-a072-b8ad585850c5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:00.328496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5c30f64f-0489-4fe8-a072-b8ad585850c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:00.337363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 774/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:00.502335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3c08-e338-4480-a97c-88ab3ab32fe6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:00.527638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3c08-e338-4480-a97c-88ab3ab32fe6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:00.535179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:00.535767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:00.536040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:00.536303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:00.545371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:00.546998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982bb27-d767-42ca-837a-9a52bfaaaff5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:00.560713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 750/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:01.372465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44826-2a99-4a78-99b5-0c4763228054 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:01.373425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44826-2a99-4a78-99b5-0c4763228054 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:01.373425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44826-2a99-4a78-99b5-0c4763228054 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:01.375644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 775/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 28 10:14:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:01.381476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29043bc-0ed1-4e5f-bad1-717353adeb6a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:01.382253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29043bc-0ed1-4e5f-bad1-717353adeb6a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:01.382429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29043bc-0ed1-4e5f-bad1-717353adeb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:01.405458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e29043bc-0ed1-4e5f-bad1-717353adeb6a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:01.406633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e29043bc-0ed1-4e5f-bad1-717353adeb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:01.415903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 751/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 28 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:01.571323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f44826-2a99-4a78-99b5-0c4763228054 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:01.589623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f44826-2a99-4a78-99b5-0c4763228054 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:01.596277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:01.596982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:01.597257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:01.597523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:01.606647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:01.608223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a12f3-5a67-491f-813f-552132a5ce35 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:01.618168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 776/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:05.295343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:05.296262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:05.296262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:05.297681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 752/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:05.303146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4f03-2bff-49d4-83a6-e2ed56df1438 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:05.303603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4f03-2bff-49d4-83a6-e2ed56df1438 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:05.303717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4f03-2bff-49d4-83a6-e2ed56df1438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:05.330938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b85e4f03-2bff-49d4-83a6-e2ed56df1438 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:05.332155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b85e4f03-2bff-49d4-83a6-e2ed56df1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:05.341294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 777/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:05.622554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:05.624928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:05.724679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014297-9acc-41c3-b621-1867e21b1399 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:05.728993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:05.729609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:05.729888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:05.730159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:05.739462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:05.740968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89272111-eae0-4327-8dfa-400c6f762d27 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:05.751196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 753/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:07.640068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:07.640968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:07.640968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:07.643278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 778/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:07.648819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366821a-6993-4603-b924-ec284f1f29a4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:07.649248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366821a-6993-4603-b924-ec284f1f29a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:07.649340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366821a-6993-4603-b924-ec284f1f29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:07.669715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1366821a-6993-4603-b924-ec284f1f29a4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:07.670926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1366821a-6993-4603-b924-ec284f1f29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:07.679336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 754/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:07.967346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:07.969262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:07.982841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231b81e-7eb3-47bc-aa8b-4e8d1da829a9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:07.986736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:07.987320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:07.987592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:07.987854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:07.996876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:07.998301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc836214-58ba-4469-b0ef-05c92c3e6916 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:08.007722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 779/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:11.277183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:11.278195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:11.278195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:11.280235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 755/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:14:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:11.289192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d5957-05d1-41ae-923b-d820666558e9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:11.289630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d5957-05d1-41ae-923b-d820666558e9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:11.290090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d5957-05d1-41ae-923b-d820666558e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:11.312531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f4d5957-05d1-41ae-923b-d820666558e9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:11.313732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f4d5957-05d1-41ae-923b-d820666558e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:11.322063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 780/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:11.873686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:11.875844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:11.958448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:14:11.973158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ac90a-967a-40cd-9ca8-ef342af970b5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:11.977206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:11.977808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:11.978085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:11.978349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:11.987686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:11.989152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a84c0-cb8b-4ceb-9a3f-4405945345f2 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:11.998575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 756/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:14:13.909881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:13.910774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:13.910774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:13.913012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 781/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 28 10:14:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:14:13.922232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3db6d-b713-4540-8a62-fe1648df1730 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:13.922675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3db6d-b713-4540-8a62-fe1648df1730 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:13.922783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3db6d-b713-4540-8a62-fe1648df1730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:13.949451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-27a3db6d-b713-4540-8a62-fe1648df1730 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:14:13.950641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27a3db6d-b713-4540-8a62-fe1648df1730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:13.960051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 757/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 28 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:14:14.522253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:14:14.525240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:14:14.544680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932dff2-71f9-45b0-a5f6-3cb08760e78d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:14:14.548851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:14:14.549467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:14:14.549747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:14:14.550003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:14:14.559994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:14:14.561473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6057bc-fe7f-45d1-811e-30b50f62cb84 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:14:14.575860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 782/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:14:23.944171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=11 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:14:24.567229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=33 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:38.620480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:38.621620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:38.622127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:38.647527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:38.648812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:38.665665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 758/1540] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:38.674968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:38.676060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:38.676638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:38.709972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:38.711802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:38.720369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 783/1541] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:38.725546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:38.744076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac2ea0-3d24-4aa0-a396-c9540d8c6131 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:38.752274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:38.752933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:38.753209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:38.753480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:38.754144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:38.754570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:38.756452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a56a8-d5b8-445f-b4b9-4645bbe4bcf7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:38.778943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 759/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:38.787200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:38.798371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec19de-6b73-4731-8fa9-d3ec46087815 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:38.813115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:38.814176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:38.814672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:38.815170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:38.816022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:38.818402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:38.883883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de8b9-49b6-4218-94a5-e76551729b3e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:38.884144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de8b9-49b6-4218-94a5-e76551729b3e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:38.884425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de8b9-49b6-4218-94a5-e76551729b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:38.889246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 784/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:38 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:38.893278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:38.922701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5a5de8b9-49b6-4218-94a5-e76551729b3e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:38.923939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5a5de8b9-49b6-4218-94a5-e76551729b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:38.991656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24c32-6aa4-4420-947f-6dfa33d4e867 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.009483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.010535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.011017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.011508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.012358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.013640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:39.013640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:15:39.013640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:15:39.015247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.041090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 760/1544] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:15:39.050114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.050433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.050653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.054261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 785/1545] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:38 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:15:39.058317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.070915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f866126-1a3f-4e28-99a2-a1b222f696a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.082552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.083487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.084096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.084624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.085504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.085629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.086750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.087834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61b467-19e3-46af-b9d0-cfdb4ec6686d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.103618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 786/1546] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:39.105581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.106064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.106678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.111286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 761/1547] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:15:39.112150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.134894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a312d-e6f0-4d7c-88bc-660aa38c07fa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.140080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.141734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.150002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.150642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.150933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.151194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.151859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.153524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ba9e3-1f42-4aa9-b85e-ba974bbfa11b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.156860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 787/1548] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.157959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.171082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 762/1549] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:39.172394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.173323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353fd21-089a-49ac-80f0-dd27137c8c3e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.180777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.181091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.182964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.183855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.184346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.184857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.185740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.186240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.187599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:39.187599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:15:39.187599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:15:39.188795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4fe5-521a-4cf8-ad65-bd0ef1ff3bce admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.201967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 788/1550] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.203677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010fc2-28e2-4c82-9b02-d5b4e8c2a8a0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.204203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010fc2-28e2-4c82-9b02-d5b4e8c2a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.204721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010fc2-28e2-4c82-9b02-d5b4e8c2a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.211598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.214492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.232720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02010fc2-28e2-4c82-9b02-d5b4e8c2a8a0 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.234232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02010fc2-28e2-4c82-9b02-d5b4e8c2a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.234878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 763/1551] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.235905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.243533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 789/1552] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.248196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.248475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.248778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.254978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbe7ea-0c6c-4c9c-946e-107b8979ba0a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.260881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.261755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.261884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.262104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] PATH_INFO: `/v3/projects/4a3dd0514b8542d280fbe2f798714ea3/users/5bb45c233ca243388bee05dede63415f/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.271969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.273216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.279508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3dd0514b8542d280fbe2f798714ea3, user_id=5bb45c233ca243388bee05dede63415f, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.280622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:39.280622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:15:39.280622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:15:39.281170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 790/1553] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.282136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.285122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a41c4-e844-40e5-8eef-a002e30dcd0e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.291786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67478dda-7f5b-4217-99f7-b533dfbc905a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.298639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 764/1554] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:39 2022] PUT /identity/v3/projects/4a3dd0514b8542d280fbe2f798714ea3/users/5bb45c233ca243388bee05dede63415f/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:39.299930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.300210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.300483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.321896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.323094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.349932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.350554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.350831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.351098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.351727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.353691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbddbf-473e-4784-82c0-aa98d299891f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.368124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:39.381356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 765/1555] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.382378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.384822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 791/1556] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:39.386002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3342cc-ce8d-42cd-a6af-951cb5ebad43 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.386217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3342cc-ce8d-42cd-a6af-951cb5ebad43 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.386485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3342cc-ce8d-42cd-a6af-951cb5ebad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.395332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b91a-2579-411d-8d17-37faec2f9544 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.406801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5f3342cc-ce8d-42cd-a6af-951cb5ebad43 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.408005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f3342cc-ce8d-42cd-a6af-951cb5ebad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.425854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:39.453636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.454259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.454537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.454802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.455535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.455837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.457481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce325f-d8c7-426c-8780-8bbcce6deb0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.469622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 766/1557] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.469823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 792/1558] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.471135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd58f-6b1c-4805-8d7b-69e7f5601b21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.471420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd58f-6b1c-4805-8d7b-69e7f5601b21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.471692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd58f-6b1c-4805-8d7b-69e7f5601b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.477964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69de7-61c1-4e3a-a389-56855abf1666 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.478371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69de7-61c1-4e3a-a389-56855abf1666 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.478701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a69de7-61c1-4e3a-a389-56855abf1666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.494074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bdefd58f-6b1c-4805-8d7b-69e7f5601b21 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.495282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bdefd58f-6b1c-4805-8d7b-69e7f5601b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.498352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-43a69de7-61c1-4e3a-a389-56855abf1666 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.499508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43a69de7-61c1-4e3a-a389-56855abf1666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.503243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 767/1559] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.504401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.504711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.504983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.515808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 793/1560] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.515808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed633d-655e-4c8b-bedb-a701e74685b0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.515808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed633d-655e-4c8b-bedb-a701e74685b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.515808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed633d-655e-4c8b-bedb-a701e74685b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.524743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.525943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.532094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b4ed633d-655e-4c8b-bedb-a701e74685b0 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.533266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b4ed633d-655e-4c8b-bedb-a701e74685b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.534912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 768/1561] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.536929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.541436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 794/1562] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.542709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f1a6-eaaf-4a9f-b925-b072bbfbfd10 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.543060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f1a6-eaaf-4a9f-b925-b072bbfbfd10 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.543396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f1a6-eaaf-4a9f-b925-b072bbfbfd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.549704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0d7c3-a267-4042-9693-1e02bbe4b574 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.554627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.555242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.555523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.555795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.556460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.558409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.563212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-60e6f1a6-eaaf-4a9f-b925-b072bbfbfd10 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.564399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60e6f1a6-eaaf-4a9f-b925-b072bbfbfd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.573344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 795/1563] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.574718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.575076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.575338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.586112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 769/1564] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:39.586992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.596900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44593-16fd-4055-b1a1-d4bf14dea817 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.597231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.598440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.601878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.602587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.602864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.603123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.603754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.604827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:39.604827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:15:39.604827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:15:39.606010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529854a6-729d-4e27-b5f7-bb9b4144c83b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.606660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 796/1565] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.607432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.616987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f776de5-28c1-40cc-af54-b22edf78da9d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.621742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.622388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.622666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.622931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.623584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.623882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.625564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.636834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 770/1566] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:39.644687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 797/1567] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.644836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.644836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.645038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.645129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.651164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db3bf9-ea1b-4030-8811-37e127fea9e0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.658727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:39.659923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:39.708158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.708882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.709122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.709407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.710046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.710347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.711992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.720087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 771/1568] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:39.729711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.729828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 798/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.729828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.733995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117b068-4379-4e9f-827a-f03239cb71d3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.738591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.739158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.739432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.739726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.740468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.740786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.742364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.754307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 799/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.761510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.771154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc1ded-6750-48fd-92a7-9f21a07f25db admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.775682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.776261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.776567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.776842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.777471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.778885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5984c-3742-4876-83ea-bea68dfc94ad None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.779051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.782442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.783044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.783323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.783599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.788783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 800/1571] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:39.789891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.793072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.794516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.798944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917917-364a-43d1-9828-8422dd674a2f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.803793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.804370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.804490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 772/1572] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:39.804669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.804939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.805191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.805574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.805874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.807467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.814745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3ea9-bd8e-4954-9847-2d4a51e82c38 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.820375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.821087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 801/1573] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.821697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.828276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.828748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.829459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.829754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.831907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.833220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27129f82-44f2-4153-9935-64077c3222d0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.838427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.839031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.839381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.839586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.840269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.842342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.851872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:39.853859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:39.859979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 773/1574] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:39.861294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.871767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc30a-389c-470d-8a72-6d6e83d20583 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.875223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 802/1575] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:39.876789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.877399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.877676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.878236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.878691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:39.878984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.879217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.880700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.894516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 774/1576] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:39.895249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:39.905610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507fdc7-0479-45c0-8a66-4299febb8e70 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.959881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56fd5b-4e2c-4c84-92b1-fc9921284450 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:39.964700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.965319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.965595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.965986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.966627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.968618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:39.970659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:39.971328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:39.971603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:39.971865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:39.972575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:39.974530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.008731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 803/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.009126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 775/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:39 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.010575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.013162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.078488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f856-6083-421f-89d2-e0b286742769 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.083434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.084635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a6a4-720e-4e9b-b2f0-4f385a295cb1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.085137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.085420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.085691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.086329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.088226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.090320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.091887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.092533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.093015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.093896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.095584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.109580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 804/1579] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.110435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.120603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688fbc0-9214-4d84-b562-f6abeb20c6ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.122504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 776/1580] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.126671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.204059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.205017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.205304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.205575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.206199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:40.206331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.209009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.211621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68a63e-b943-4c31-a8a6-c5cd6eba40e6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.219041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.219804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.220177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.220719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.221465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.225195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.225345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 805/1581] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.226266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.238235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4558-a7bf-46b0-bf7b-a9ad73f93c3f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.243029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.243641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.243915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.244184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.244858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.246945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.251885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:40.252577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 777/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.253262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.264465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de46b-f99c-4ad9-a2c5-58dcb0d5048d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.269838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.270501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.270858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.271155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.271868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.274039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.286164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 806/1583] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.286164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.305759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 778/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.305759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.314629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8850a-48c3-4a30-a00f-e5401822941b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.356004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cfcf-3bef-431a-a959-f6ecc289db14 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.361019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.362373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.362679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.362948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.363634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.365636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.382443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.383265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.383679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.384118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] PATH_INFO: `/v3/projects/7d83a21ea63d41909cb369f94cf3a3cc/users/af1daf846ccd46f1819ba0010875c33b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.392686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 807/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.395531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.401657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d83a21ea63d41909cb369f94cf3a3cc, user_id=af1daf846ccd46f1819ba0010875c33b, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.404633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.411032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00270da-4031-4339-8ba6-8573e4d16b54 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.420080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.421077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.421360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.421737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.422356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.432623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 779/1586] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:40 2022] PUT /identity/v3/projects/7d83a21ea63d41909cb369f94cf3a3cc/users/af1daf846ccd46f1819ba0010875c33b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:40.432820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.432924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.464155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 808/1587] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.465266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.477398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f8cae-8b30-4577-b78a-40ee107b0c55 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.513594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b2cb9-d29f-4cc2-b223-9f417117b16b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.520383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.521127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.521357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.521617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.522293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.523905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.537789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 780/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.538656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.549532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28079-86e1-4bac-bcfb-e8da8b6c0591 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.555622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.556748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.557264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.557773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.558605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.560820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.562092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.562862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.563151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.563427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.564461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.566235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.581706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 809/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.581706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.596417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30478d-0380-43f8-8d65-99679b85a4f5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.609063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 781/1590] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/users => generated 296 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:40.610387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.611041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.611625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.611910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.612182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.612907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.614521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c514c4-ed16-48a4-bc0d-0e83ea6ad737 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.623688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ba130-57e3-4008-b364-c05e2587c3e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.627195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 810/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.628751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.629132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.629638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.630193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.631305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.631589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.631946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.632748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.635001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.648651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 782/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.649447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.660698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc54e-6a25-4a83-b702-56c10f292026 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.665998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.666751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.666895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.667155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.667804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.668736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] MFA Rules not processed for user `af1daf846ccd46f1819ba0010875c33b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:40.670223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:40.672856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.694247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 783/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.695138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.706181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cbcc4-8409-4228-b677-76ad73868bc8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.707615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:40.713267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.714106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.714387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.714662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] PATH_INFO: `/v3/projects/eefcb341ab8d4823a95c5bce47395083/users/ed4711b7691643109007e173fee1c405/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.718677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:40.735245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eefcb341ab8d4823a95c5bce47395083, user_id=ed4711b7691643109007e173fee1c405, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.738447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.758724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 784/1594] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:40 2022] PUT /identity/v3/projects/eefcb341ab8d4823a95c5bce47395083/users/ed4711b7691643109007e173fee1c405/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:40.758724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.759537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 811/1595] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:40.760676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.776093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d5a11-518f-4594-a2d5-9759a4a9d94e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.857449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.858383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.858808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.859313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.860125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.862386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.870000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a00db-5b2c-4fd8-ae1d-f5a0bb2b3140 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.875760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.879190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 812/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.879351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.879351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.879548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.879648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.879648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.879814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.891410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d229d3-813f-412c-a7dc-7aa8ec8684c8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.892024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 785/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.892918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.897718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.898505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.898968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.899278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.900209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.902245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.907218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4e2b-1931-436b-bd23-92bd2d7f5755 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.913097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.913713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.914336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.914604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.918003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 813/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.918160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.918246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.918331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23ee46-25fd-4452-b636-afe4de43aa80 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.932234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 786/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.932234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.932234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.932234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.932725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1dcee-924f-4817-adb5-48f3f39f1ed9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.940154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:40.942278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.943311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.943752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.944796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:40.948598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654ebd3-e9a7-4293-8771-b5e6f179b8d8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:40.957827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:40.959349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:40.969767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 814/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:40.971309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570d1a-b1c7-4dbd-a56e-373e0e8e8f50 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:40.971647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570d1a-b1c7-4dbd-a56e-373e0e8e8f50 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:40.971927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1570d1a-b1c7-4dbd-a56e-373e0e8e8f50 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:40.973441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 815/1601] 10.222.0.40 () {54 vars in 962 bytes} [Sun Aug 28 10:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:40.973905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 787/1602] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:40.977321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.978999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1570d1a-b1c7-4dbd-a56e-373e0e8e8f50 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:40.996541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1570d1a-b1c7-4dbd-a56e-373e0e8e8f50 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:40.996847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1185d-951c-4205-8d84-b53c6bac0908 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.002012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.002875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.003200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.003654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.004142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.005154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.005441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.005779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.005988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] PATH_INFO: `/v3/projects/d88cb2f7d9d243eab09d20334b1e0044/users/555e8ced657b48fc8b6f38d00403008d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.007898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.020827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 816/1603] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:41.022278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.025316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88cb2f7d9d243eab09d20334b1e0044, user_id=555e8ced657b48fc8b6f38d00403008d, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.029635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.034779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738d4c-211c-4dfe-8839-45b77a8ac133 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.042716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.044169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.044452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.044849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] PATH_INFO: `/v3/projects/f186d5bdc9b84b4aaae07b15f7cf132f/users/d39e2acbd72a42758f63535bac526248/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.052759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 788/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:40 2022] PUT /identity/v3/projects/d88cb2f7d9d243eab09d20334b1e0044/users/555e8ced657b48fc8b6f38d00403008d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:41.054067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.068624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f186d5bdc9b84b4aaae07b15f7cf132f, user_id=d39e2acbd72a42758f63535bac526248, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.072254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.095988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 817/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:41 2022] PUT /identity/v3/projects/f186d5bdc9b84b4aaae07b15f7cf132f/users/d39e2acbd72a42758f63535bac526248/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:41.097216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.153529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461cf3-75f8-43b4-98e1-b41255020a61 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.200605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8196-5bb3-4e0c-b585-75b33ac02b6a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.205611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.206105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.206384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.206653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] PATH_INFO: `/v3/projects/ac2672bc171e4c64aee4ae62797924d9/users/824071fbc0bc44c7ab673394d1ee0b5d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.214270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:41.232606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2672bc171e4c64aee4ae62797924d9, user_id=824071fbc0bc44c7ab673394d1ee0b5d, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.232943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.233723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.234102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.234473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] PATH_INFO: `/v3/projects/98de7ba1be02492b9a24eb3bd17c1192/users/807514ea5ca9442e986faf5318253bc8/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.235286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbfc66-794d-4871-addd-c22c73f93195 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.237622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:41.251144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=98de7ba1be02492b9a24eb3bd17c1192, user_id=807514ea5ca9442e986faf5318253bc8, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.254017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.255650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 818/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:41 2022] PUT /identity/v3/projects/ac2672bc171e4c64aee4ae62797924d9/users/824071fbc0bc44c7ab673394d1ee0b5d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:41.257538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.258095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.258532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.276455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 789/1607] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:41 2022] PUT /identity/v3/projects/98de7ba1be02492b9a24eb3bd17c1192/users/807514ea5ca9442e986faf5318253bc8/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:41.277694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.316475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.318010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:41.384650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8eefab-1ab8-4f72-923f-ac59ee7af9cd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.391172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.391791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.392194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.392340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.424936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 819/1608] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:41.425809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.476351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.478177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dc29b-eba4-446c-b1f5-010ccceba3c2 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.496157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 790/1609] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:41.497777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfe362-af31-4973-8b1f-ae2de5cf0a5e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.498381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfe362-af31-4973-8b1f-ae2de5cf0a5e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.498839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfe362-af31-4973-8b1f-ae2de5cf0a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.533242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1ccfe362-af31-4973-8b1f-ae2de5cf0a5e None None] MFA Rules not processed for user `555e8ced657b48fc8b6f38d00403008d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.534862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1ccfe362-af31-4973-8b1f-ae2de5cf0a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:41.542902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78f0f8-dc6d-48e4-bcbf-297a59600a57 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.554411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.557364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.558021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.558450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] PATH_INFO: `/v3/projects/76d489fb3d144cc3abf03a842e70636d/users/edbf26fde0254259a540f31139fc1a51/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.581852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d489fb3d144cc3abf03a842e70636d, user_id=edbf26fde0254259a540f31139fc1a51, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:41.583449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684f476-2442-4463-a635-8dbd3bc8cfbf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:41.691558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 820/1610] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:41 2022] PUT /identity/v3/projects/76d489fb3d144cc3abf03a842e70636d/users/edbf26fde0254259a540f31139fc1a51/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:41.691558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb62d-b467-4c94-8d93-07948bce525f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.691558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb62d-b467-4c94-8d93-07948bce525f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.691558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb62d-b467-4c94-8d93-07948bce525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.691962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 791/1611] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:41.691962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.691962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.691962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.693178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ce6bb62d-b467-4c94-8d93-07948bce525f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.693178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce6bb62d-b467-4c94-8d93-07948bce525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:41.696957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:41.723674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] MFA Rules not processed for user `824071fbc0bc44c7ab673394d1ee0b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.725213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:41.780399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:41.804020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 821/1612] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:41.806216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e947f8-615a-4a8d-9892-26ffa0428073 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.807734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e947f8-615a-4a8d-9892-26ffa0428073 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.807952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e947f8-615a-4a8d-9892-26ffa0428073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.845982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 792/1613] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:41.846708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:41.858717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b4e947f8-615a-4a8d-9892-26ffa0428073 None None] MFA Rules not processed for user `807514ea5ca9442e986faf5318253bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.860141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b4e947f8-615a-4a8d-9892-26ffa0428073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:41.961875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 822/1614] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:41.964078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.964463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.964909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.966753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44191c-0939-4136-b08a-0eaad1f306e7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:41.975394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:41.976030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:41.977521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:41.977806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:41.990342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:41.992193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:42.003223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 823/1615] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:42.004301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.016378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da014ca1-d7f6-4332-9c93-bff49b7f1e5d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.037275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.038145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.039406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.039684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.041735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.055076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.058279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.060266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.080200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 793/1616] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:42.081445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.114220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 824/1617] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:42.115781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.133301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:42.153229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f825-4f29-4d3e-b397-dd90f2f311dc service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.220751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.221293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.221687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.221966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.266637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6941b4-c5d4-484c-b27d-2509d46ceeed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.293414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.294155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.296104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.297283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.300120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.305928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.317915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.319756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.342504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 794/1618] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:42.343565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.388566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 825/1619] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:42 2022] POST /identity/v3/projects => generated 383 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:42.388566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.426794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8e563-978d-4504-8872-56d796cb0daa service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.506041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.506701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.506985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.507256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.518453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:42.525333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:42.565972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9fe79-a435-4fd3-8b20-8b69d0152747 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.589675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.591674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.595879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.596293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.597634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.602828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.622618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 795/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:42.624202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.719209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68518b4-79ec-4e26-ba1d-62b113de7479 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.727057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.727766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.728255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.728816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.739052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.747039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.790959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.793867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.806460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 796/1621] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:42 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:42.806460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.828393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 826/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:42.835460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.838231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0d7d-3e35-4d32-b709-6d77591b08e5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.852759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.854719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.855404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.855693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.856360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.862376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.873446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b92e3-b0ce-4097-a388-c3fe94d8c966 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.888219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.888976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.889250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.894184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.896830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.904983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.919968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 797/1623] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:42 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:42.924251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.947398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 827/1624] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:42.948756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:42.950546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525cdca-2c5e-4b3d-8f54-249374e259f7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.967712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13340-8068-4b7e-aa0d-65474b56aa88 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:42.968944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.970336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.970336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.970628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:42.971286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:42.975341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:42.982066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:42.982799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:42.984122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:42.984415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] PATH_INFO: `/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a/users/5a930fd048c4452ab4a02d089762b99a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.015433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 798/1625] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:43.017001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c9a9b5ff2445d4b85bc584d2b0602a, user_id=5a930fd048c4452ab4a02d089762b99a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.019993 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.021014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:43.046317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 828/1626] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:42 2022] PUT /identity/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a/users/5a930fd048c4452ab4a02d089762b99a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:15:43.049688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:43.061037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d6dda-9fcb-4aae-8c02-be16651921b2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:43.225494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:43.226184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.226466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.226737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] PATH_INFO: `/v3/projects/22bc5e0b440742febe5afd053bf144b8/users/c5a8bab7e3c645768c30dc19ac38bc7a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.230833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa899-56ed-4bfe-a906-b313e070d74f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:43.245110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:43.247550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.248793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.249200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.249964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.255326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1673b-9694-444b-ad23-8cb2077258e8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.263357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22bc5e0b440742febe5afd053bf144b8, user_id=c5a8bab7e3c645768c30dc19ac38bc7a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.270581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ce18-bfba-4660-b25b-721a183f9b06 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.277629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:43.281948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 829/1627] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:43.284889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd6ea-9cee-4ec2-b06a-c15898811e2a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.285471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd6ea-9cee-4ec2-b06a-c15898811e2a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.285846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd6ea-9cee-4ec2-b06a-c15898811e2a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.290939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 830/1628] 10.222.0.40 () {54 vars in 962 bytes} [Sun Aug 28 10:15:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:43.292959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bd6ea-9cee-4ec2-b06a-c15898811e2a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:43.301815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 799/1629] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:43 2022] PUT /identity/v3/projects/22bc5e0b440742febe5afd053bf144b8/users/c5a8bab7e3c645768c30dc19ac38bc7a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:43.303571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1a09-15c7-4f12-a706-03357cd65c74 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.304096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1a09-15c7-4f12-a706-03357cd65c74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.304832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1a09-15c7-4f12-a706-03357cd65c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.346633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eaad1a09-15c7-4f12-a706-03357cd65c74 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:43.347917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eaad1a09-15c7-4f12-a706-03357cd65c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:43.411975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:43.455388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bd6ea-9cee-4ec2-b06a-c15898811e2a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:43.464195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:43.465728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.466191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.466554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] PATH_INFO: `/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a/users/5a930fd048c4452ab4a02d089762b99a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.469177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c9a9b5ff2445d4b85bc584d2b0602a, user_id=5a930fd048c4452ab4a02d089762b99a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.472560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.491006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 831/1630] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:43 2022] PUT /identity/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a/users/5a930fd048c4452ab4a02d089762b99a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:43.502065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:43.563590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 800/1631] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Aug 28 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:43.572423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa95e9-e7db-46c0-a49e-0ac14f18e7bb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.572855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa95e9-e7db-46c0-a49e-0ac14f18e7bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.573127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa95e9-e7db-46c0-a49e-0ac14f18e7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.628408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ff52-b759-4e62-b699-51029673d8f3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:43.638207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:43.639003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.639281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.639555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.640213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.648154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b51447-fa5e-461e-9783-5513cdb2801e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.678759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 832/1632] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:43.678759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc8fc-7623-47ce-8b35-4b53fdff224a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.679496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc8fc-7623-47ce-8b35-4b53fdff224a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.679694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc8fc-7623-47ce-8b35-4b53fdff224a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.685951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 833/1633] 10.222.0.40 () {54 vars in 912 bytes} [Sun Aug 28 10:15:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:43.686726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fc8fc-7623-47ce-8b35-4b53fdff224a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:43.702154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1aaa95e9-e7db-46c0-a49e-0ac14f18e7bb None None] MFA Rules not processed for user `5a930fd048c4452ab4a02d089762b99a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:43.706167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1aaa95e9-e7db-46c0-a49e-0ac14f18e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:43.734857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fc8fc-7623-47ce-8b35-4b53fdff224a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:43.739936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:43.741209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.741657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.742087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] PATH_INFO: `/v3/projects/22bc5e0b440742febe5afd053bf144b8/users/c5a8bab7e3c645768c30dc19ac38bc7a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.744598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22bc5e0b440742febe5afd053bf144b8, user_id=c5a8bab7e3c645768c30dc19ac38bc7a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:43.747799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245f808-2a95-4966-abb7-25a489325f63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:43.760875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 834/1634] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:43 2022] PUT /identity/v3/projects/22bc5e0b440742febe5afd053bf144b8/users/c5a8bab7e3c645768c30dc19ac38bc7a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:43.763075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.763562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.764127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:43.813286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:43.814642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:43.933980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 801/1635] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:43.938598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:43.939030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:43.939494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.005309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] MFA Rules not processed for user `c5a8bab7e3c645768c30dc19ac38bc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:44.009568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:44.017264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 835/1636] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:44.019590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:44.179892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5efa64-f1b8-44a8-a23a-1a07d69629ef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:44.187119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:44.189438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:44.190847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.191132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.191404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.202337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 802/1637] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:44.203217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:44.224225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:44.229197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e35c-859c-4aff-adcd-ed5cfe345377 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:44.236882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:44.239942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.240384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.240693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.266261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:44.267735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:44.293320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 803/1638] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:44.310913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:44.312584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955292-a1b4-4fec-8bc0-99ab7ebb67b4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:44.323856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 836/1639] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:44.401808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:44.414990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2777-1d5c-4d57-b58a-c8a6f417a3dd service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:44.419052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:44.419754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.421195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.421560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.433217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:44.434981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8be00a-87fc-458a-931c-3c723fc70a99 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:44.461300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 804/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:44.492918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e86e4-1286-461f-9bd3-fb48f80ee597 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.493506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e86e4-1286-461f-9bd3-fb48f80ee597 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.493953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e86e4-1286-461f-9bd3-fb48f80ee597 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.495789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 837/1641] 10.222.0.40 () {54 vars in 912 bytes} [Sun Aug 28 10:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:44.502007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.502498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.502917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.558497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:44.566573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:44.612204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 805/1642] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:44.636345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e86e4-1286-461f-9bd3-fb48f80ee597 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:44.657313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e86e4-1286-461f-9bd3-fb48f80ee597 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:44.670243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:44.670879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.672674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.672938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.714927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:44.716713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:44.751422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 838/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:44.824043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:44.842142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3078db-d33a-4a86-b5b3-9202d39d0639 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:44.848019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:44.848808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:44.849378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:44.850136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:44.990562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:44.994059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82915ea-5d8b-4a8e-b216-2fe447216a4b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:45.035096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 806/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:45.153337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:45.361444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d4fe4-aa66-4a51-a45e-6413d1a8f546 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:45.371931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:45.372917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:45.375104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:45.375552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] PATH_INFO: `/v3/users/af1daf846ccd46f1819ba0010875c33b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:45.377261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1daf846ccd46f1819ba0010875c33b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:45.382547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:45.487707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1daf846ccd46f1819ba0010875c33b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.506917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1daf846ccd46f1819ba0010875c33b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.546042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1daf846ccd46f1819ba0010875c33b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.573044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1daf846ccd46f1819ba0010875c33b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.574329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1daf846ccd46f1819ba0010875c33b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.602083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 839/1645] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:15:45 2022] DELETE /identity/v3/users/af1daf846ccd46f1819ba0010875c33b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:45.606876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:45.607215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:45.607496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:45.647597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:45.650083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:45.677536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:45.680150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:45.719222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:45.806899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398373ff-dcbb-4470-9003-69e12704dc41 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:45.809995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 807/1646] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:45.819168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:45.821014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:45.824913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:45.825600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:45.825883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:45.826154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] PATH_INFO: `/v3/users/824071fbc0bc44c7ab673394d1ee0b5d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:45.830038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=824071fbc0bc44c7ab673394d1ee0b5d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:45.836240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:45.955872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824071fbc0bc44c7ab673394d1ee0b5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.964072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967e85-1d5a-4265-bfcc-54b666167dd6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:45.969038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:45.969733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:45.970172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:45.970458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:45.984116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:45.986028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824071fbc0bc44c7ab673394d1ee0b5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:45.986889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902fe9c-9cb4-442b-a63a-5982ee917381 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:46.000956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 808/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:46.026400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824071fbc0bc44c7ab673394d1ee0b5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:46.058016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824071fbc0bc44c7ab673394d1ee0b5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:46.058865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824071fbc0bc44c7ab673394d1ee0b5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:46.070906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 840/1648] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:15:45 2022] DELETE /identity/v3/users/824071fbc0bc44c7ab673394d1ee0b5d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:15:46.072653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:46.072947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:46.073223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:46.100236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:46.115075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:46.116365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:46.283583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 809/1649] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:46.353446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:46.355262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:46.609325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb8a40-7e18-49bc-bed1-231e765344fe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:46.615055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:46.616190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:46.616916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:46.617382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:46.632283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:46.634371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524387c-1d98-4b5c-875a-af64f620957a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:46.654120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 841/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:47.731697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:47.733397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:47.752255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034ea49-94c3-4355-9bad-3efc8a7c6acb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:47.763157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:47.763836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:47.764117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:47.764392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] PATH_INFO: `/v3/projects/7d83a21ea63d41909cb369f94cf3a3cc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:47.769344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d83a21ea63d41909cb369f94cf3a3cc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:47.771359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:47.884133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d83a21ea63d41909cb369f94cf3a3cc', 'request_id': 'req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:47.899575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d83a21ea63d41909cb369f94cf3a3cc', 'request_id': 'req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:47.909730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d83a21ea63d41909cb369f94cf3a3cc', 'request_id': 'req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:47.943632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] The token cache is being invalidate because project 7d83a21ea63d41909cb369f94cf3a3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:15:47.943931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:47.947454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:47.949119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 810/1651] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:15:47 2022] DELETE /identity/v3/projects/7d83a21ea63d41909cb369f94cf3a3cc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:47.954094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e975-ec64-4265-aed7-46831575ff2a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:47.954937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e975-ec64-4265-aed7-46831575ff2a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:47.955365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e975-ec64-4265-aed7-46831575ff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:47.996312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-96f6e975-ec64-4265-aed7-46831575ff2a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:47.997753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-96f6e975-ec64-4265-aed7-46831575ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:48.119132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:48.156387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.158316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:48.199842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 842/1652] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:48.201194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.208353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.212281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.269782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:48.271108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:48.311995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 843/1653] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:48.316380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.356146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f494652-9ceb-4efc-9f7f-b6067da019c7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.377792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.378443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.378722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.378992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.379660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:48.379965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.383456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.415576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488d1d3-8888-4ce3-84e7-aa6a78be5bd0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.432182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.432942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.433223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.433495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] PATH_INFO: `/v3/projects/ac2672bc171e4c64aee4ae62797924d9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.435617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2672bc171e4c64aee4ae62797924d9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.448195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 844/1654] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:48.449212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.450069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.481180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ced6a-6607-49aa-8269-df9e0eb14007 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.514004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.514656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.514936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.515207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.515854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.517824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.568542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:48.605226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2672bc171e4c64aee4ae62797924d9', 'request_id': 'req-7031ba53-3c53-43fd-94ff-9b152af6409b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:48.605813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 845/1655] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:48 2022] POST /identity/v3/projects => generated 357 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:48.611249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.621939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2672bc171e4c64aee4ae62797924d9', 'request_id': 'req-7031ba53-3c53-43fd-94ff-9b152af6409b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:48.648912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2672bc171e4c64aee4ae62797924d9', 'request_id': 'req-7031ba53-3c53-43fd-94ff-9b152af6409b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:48.685395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] The token cache is being invalidate because project ac2672bc171e4c64aee4ae62797924d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:15:48.685702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:48.688690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:48.694696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 811/1656] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:15:48 2022] DELETE /identity/v3/projects/ac2672bc171e4c64aee4ae62797924d9 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:15:48.709298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.718907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:48.805928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4069994-6b4c-416a-865f-078c9511c144 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.820487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.821152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.821433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.821707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.822439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.825917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.837238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031ba53-3c53-43fd-94ff-9b152af6409b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.846418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.847094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.847383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.847659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] PATH_INFO: `/v3/users/c5a8bab7e3c645768c30dc19ac38bc7a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.848932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a8bab7e3c645768c30dc19ac38bc7a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.850229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:48.850229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Aug 28 10:15:48.850229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: Aug 28 10:15:48.851536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.862341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 846/1657] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:48 2022] POST /identity/v3/users => generated 279 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:48.867406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.877774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:48.912384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220cd21-21cf-4609-9753-6da3c4fab137 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:48.927615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:48.928267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:48.928650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:48.929154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:48.929826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:48.933665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:48.958517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 847/1658] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:48.966046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:48.981973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d65d4-9989-48ba-aa4a-8be1eb8e1267 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.012438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.014033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.014521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.015099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.017815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.020896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:49.035829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a8bab7e3c645768c30dc19ac38bc7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:49.054452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 848/1659] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:49.055427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:49.064785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a8bab7e3c645768c30dc19ac38bc7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:49.066374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a8bab7e3c645768c30dc19ac38bc7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:49.078035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ddf33-ff2f-4e53-bbfa-65706a86d0c1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.084469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.086569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a8bab7e3c645768c30dc19ac38bc7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:49.087405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f42ec816-5f9e-48b5-bbb3-3b56df93558a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a8bab7e3c645768c30dc19ac38bc7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:49.094560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.095475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.097024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] PATH_INFO: `/v3/projects/2df4ce3a5f7e4dbb95d16e3f7bdae142/users/c8d1a4630510480ab8deb3f30ffbb83c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.132411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df4ce3a5f7e4dbb95d16e3f7bdae142, user_id=c8d1a4630510480ab8deb3f30ffbb83c, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.138618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:49.143706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 812/1660] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:15:48 2022] DELETE /identity/v3/users/c5a8bab7e3c645768c30dc19ac38bc7a => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:49.165252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 849/1661] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:49 2022] PUT /identity/v3/projects/2df4ce3a5f7e4dbb95d16e3f7bdae142/users/c8d1a4630510480ab8deb3f30ffbb83c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:49.177782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.178085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.178322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.255227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] MFA Rules not processed for user `c8d1a4630510480ab8deb3f30ffbb83c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:49.260148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:49.323558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:49.440624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 813/1662] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:49.445040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:49.447589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:49.623670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:49.657163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757b18-6831-4705-a664-3205615590e3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.667264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.667914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.668196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.668466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.669196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.674573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:49.720272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 850/1663] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:49 2022] POST /identity/v3/projects => generated 357 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:49.724322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:49.741722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185edc8-d45b-41b8-b3d6-f8a45bf1f065 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.836450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.837305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.837910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.837910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.838894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.841217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:49.888613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 814/1664] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:49 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:49.889150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:49.904887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c5ec6-fb38-4006-9717-b7db6c7aa81c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.911314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.912246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.913049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.913567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.916000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.919196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:49.942199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 851/1665] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:15:49.946865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:49.959138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efed47b-b630-4acf-9e92-9dd4dc473d5a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:49.966737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:49.967664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:49.968094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:49.968522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] PATH_INFO: `/v3/projects/621c25f156754c9593ca7d67512b28a0/users/d6427967e2234a189fdcd44a7f802942/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:49.988489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621c25f156754c9593ca7d67512b28a0, user_id=d6427967e2234a189fdcd44a7f802942, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:49.992444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:50.024118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 815/1666] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:49 2022] PUT /identity/v3/projects/621c25f156754c9593ca7d67512b28a0/users/d6427967e2234a189fdcd44a7f802942/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:50.028331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:50.043203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16cc1f-4120-4fdb-86e3-bd90747a81df admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:50.167199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:50.168734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.169207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.169902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:50.170772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:50.172745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de147d-19e3-44df-93f6-29864583f439 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:50.189087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 852/1667] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:15:50.195002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:50.209620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2faf6-7629-4890-a86e-9db9cf25e416 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:50.223686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:50.224441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.224764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.225703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] PATH_INFO: `/v3/projects/621c25f156754c9593ca7d67512b28a0/users/d6427967e2234a189fdcd44a7f802942/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:50.233966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621c25f156754c9593ca7d67512b28a0, user_id=d6427967e2234a189fdcd44a7f802942, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:50.236856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:50.259857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 816/1668] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:50 2022] PUT /identity/v3/projects/621c25f156754c9593ca7d67512b28a0/users/d6427967e2234a189fdcd44a7f802942/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:50.270764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.271261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.271629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:50.336228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] MFA Rules not processed for user `d6427967e2234a189fdcd44a7f802942`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:50.341654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:50.480457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 853/1669] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:50.494770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:50.496907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:50.589645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcefe5d-3bc6-424f-8441-39091ba285e0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:50.595887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:50.596787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.597459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.597894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:50.631616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:50.636460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:50.651540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 817/1670] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:50.758132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:50.759724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:50.862347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:50.909220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c21aab-9fc4-4aa0-98b6-9ea31d731b9f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:50.918846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:50.919849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.920124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.920518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] PATH_INFO: `/v3/projects/22bc5e0b440742febe5afd053bf144b8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:50.922010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22bc5e0b440742febe5afd053bf144b8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:50.923220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 28 10:15:50.923220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Aug 28 10:15:50.923220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: Aug 28 10:15:50.926093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:50.960855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:50.971818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807450-9102-444b-87a4-aa30b562569d service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:50.976958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:50.977588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:50.977869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:50.978140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.013391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:51.068890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22bc5e0b440742febe5afd053bf144b8', 'request_id': 'req-cb567d45-95a1-4f63-9b89-154ccae51c1a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:51.087205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.089350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48d2b5-fa3c-4f6c-ad0a-aec1d046d283 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.089619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22bc5e0b440742febe5afd053bf144b8', 'request_id': 'req-cb567d45-95a1-4f63-9b89-154ccae51c1a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:51.104393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 818/1671] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:51.106115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.106651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.106927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.120445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22bc5e0b440742febe5afd053bf144b8', 'request_id': 'req-cb567d45-95a1-4f63-9b89-154ccae51c1a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:51.144530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] MFA Rules not processed for user `5bb45c233ca243388bee05dede63415f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:51.146300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] The token cache is being invalidate because project 22bc5e0b440742febe5afd053bf144b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:15:51.146412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:51.146644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:51.147251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cb567d45-95a1-4f63-9b89-154ccae51c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:15:51.149655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 854/1672] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:15:50 2022] DELETE /identity/v3/projects/22bc5e0b440742febe5afd053bf144b8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:15:51.155001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ae6e1-a4d2-4166-a84b-85d7a4f45230 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.155278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ae6e1-a4d2-4166-a84b-85d7a4f45230 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.155554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ae6e1-a4d2-4166-a84b-85d7a4f45230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.220026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b03ae6e1-a4d2-4166-a84b-85d7a4f45230 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:51.221316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b03ae6e1-a4d2-4166-a84b-85d7a4f45230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:51.324618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 855/1673] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:51.332582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.332881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.333149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.354571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 819/1674] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:51.370166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:51.371636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:51.413203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:51.423176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:51.450964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 856/1675] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:15:51.455112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:51.500027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0b687-e1a3-4859-8e48-c9f9f5c9dd97 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:51.523353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:51.524392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.526666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.526959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.527796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:15:51.528109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.530115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.571769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 857/1676] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:15:51.572339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:51.601854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6fea6-56b2-47fe-89ad-5bbaf9b54980 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:51.626934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:51.627598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.627799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.628222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.629050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.633578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.646282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5ade-c8ce-47fc-9907-b5c0bc6cf3ab None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:51.669591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:51.670237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.670565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.670835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.692454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.694425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.705452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 858/1677] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:15:51 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:15:51.710321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:51.716840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 820/1678] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:15:51.788843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:51.856555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d85e1-18ff-47b5-a755-68874525565b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:51.865355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:51.866171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.866519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.866714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.867368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.870860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.923358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 859/1679] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:15:51 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:15:51.925161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:51.943139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9522d2-6114-407c-a03f-24ba74be6b87 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:51.947935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:51.950803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:51.950803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:51.951043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:51.951578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:51.956200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:51.982279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 821/1680] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:51.986701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:52.011595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6fe52-fde2-4fcf-8de8-70e294d69d7b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:52.026982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:52.027796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:52.028215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:52.030217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:52.031173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:52.034609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e082580-b81d-499a-ad2f-0893b000ecca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:52.058020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 860/1681] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:15:52.062587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:52.077268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4a72-41c7-4c56-9ed9-0c3b21d3bdc7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:52.088219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:52.088957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:52.089238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:52.089511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] PATH_INFO: `/v3/projects/a7c28643a59f479b8ebda21d3d2c1df5/users/2cb6d63eadc14cff8515bc3ed8da99bc/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:52.128566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c28643a59f479b8ebda21d3d2c1df5, user_id=2cb6d63eadc14cff8515bc3ed8da99bc, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:52.136433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:52.165549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 822/1682] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:15:52 2022] PUT /identity/v3/projects/a7c28643a59f479b8ebda21d3d2c1df5/users/2cb6d63eadc14cff8515bc3ed8da99bc/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:15:52.176068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:52.176985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:52.177438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:52.226777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] MFA Rules not processed for user `2cb6d63eadc14cff8515bc3ed8da99bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:52.228195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:52.336431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 861/1683] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:15:52.351493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:52.354850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:52.361032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:52.510570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff90612-11e7-4842-bed1-0d6aa4b1fae9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:52.525563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:52.528946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:52.529203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:52.529575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:52.558303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:52.562535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123dc3-5b42-41a4-8cd3-44564947f060 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:52.594707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 823/1684] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:53.831482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:53.904086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f8e9e-0d64-42f7-b1de-b5f54a523601 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:53.921204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:53.921848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:53.922129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:53.922898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:53.926749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:54.137479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:54.138973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:54.157738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 862/1685] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:54.393484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9115-9e5b-4110-a609-c841036cb25a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:54.394391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9115-9e5b-4110-a609-c841036cb25a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:54.394815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9115-9e5b-4110-a609-c841036cb25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:54.455071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f4cf9115-9e5b-4110-a609-c841036cb25a None None] MFA Rules not processed for user `ed4711b7691643109007e173fee1c405`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:54.456424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4cf9115-9e5b-4110-a609-c841036cb25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:54.567578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 824/1686] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:54.583324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:54.584873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:54.663820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:54.759742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3e013-125a-4001-a0e6-6a9061b96e0e service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:54.771229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:54.771901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:54.772171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:54.772433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:54.808346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:54.810450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4314c2-8316-4497-af3a-108b2b94d1cd service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:54.839366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 863/1687] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:56.921065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:56.921603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:56.921603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.037685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] MFA Rules not processed for user `d39e2acbd72a42758f63535bac526248`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:57.042909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:57.170699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 825/1688] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:57.181186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce089da-bd87-44e6-ad8f-d3d152012f41 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:57.182075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce089da-bd87-44e6-ad8f-d3d152012f41 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:57.182493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce089da-bd87-44e6-ad8f-d3d152012f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.219724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bce089da-bd87-44e6-ad8f-d3d152012f41 None None] MFA Rules not processed for user `d39e2acbd72a42758f63535bac526248`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:57.221098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bce089da-bd87-44e6-ad8f-d3d152012f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:57.258367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 864/1689] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:57.270951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:57.292725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ea51a-ba52-4013-8bb4-a24016f5bf9a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:57.317474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:57.318189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:57.318473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:57.318745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.332978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f12153-cb61-4f09-a449-6f084527e0f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:57.333334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f12153-cb61-4f09-a449-6f084527e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:57.333609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f12153-cb61-4f09-a449-6f084527e0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.362841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:57.365966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93e67d-ed77-413a-8543-b54b3105ef64 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:57.376878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b2f12153-cb61-4f09-a449-6f084527e0f6 None None] MFA Rules not processed for user `5bb45c233ca243388bee05dede63415f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:57.378115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2f12153-cb61-4f09-a449-6f084527e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:57.379499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 826/1690] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:57.410084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32bdc-bd03-451c-acf8-7cdaa01ab69d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:57.410384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32bdc-bd03-451c-acf8-7cdaa01ab69d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:57.410713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32bdc-bd03-451c-acf8-7cdaa01ab69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.477883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:57.519914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-82c32bdc-bd03-451c-acf8-7cdaa01ab69d None None] MFA Rules not processed for user `edbf26fde0254259a540f31139fc1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:57.520530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 865/1691] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:57.521241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82c32bdc-bd03-451c-acf8-7cdaa01ab69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:57.762405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 827/1692] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:57.764450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a5ae-1ef1-49d9-9d2f-23d523ee8c16 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:57.764866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a5ae-1ef1-49d9-9d2f-23d523ee8c16 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:57.765234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a5ae-1ef1-49d9-9d2f-23d523ee8c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:57.806615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2dc6a5ae-1ef1-49d9-9d2f-23d523ee8c16 None None] MFA Rules not processed for user `edbf26fde0254259a540f31139fc1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:57.811084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2dc6a5ae-1ef1-49d9-9d2f-23d523ee8c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:57.829768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 866/1693] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:58.321386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.323805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.324983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.327623 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 828/1694] 10.222.0.40 () {54 vars in 912 bytes} [Sun Aug 28 10:15:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:58.333358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2588-5624-401e-be32-c011bc5ffc54 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.333847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2588-5624-401e-be32-c011bc5ffc54 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.333940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2588-5624-401e-be32-c011bc5ffc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.406422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-17ab2588-5624-401e-be32-c011bc5ffc54 None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:58.407681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17ab2588-5624-401e-be32-c011bc5ffc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:58.522104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 867/1695] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:58.539537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:58.541889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:58.568822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d632f1-7d3d-4b71-bf86-33144cc8dfe4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:58.576668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:58.577306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.577587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.577862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.585686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84481d44-5bf9-4195-b3ab-963e5b232397 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.586028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84481d44-5bf9-4195-b3ab-963e5b232397 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.586231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84481d44-5bf9-4195-b3ab-963e5b232397 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.589739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 868/1696] 10.222.0.40 () {54 vars in 912 bytes} [Sun Aug 28 10:15:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:58.596859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d87f-4f2c-4ff3-b8db-b645ed1744cf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.597142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d87f-4f2c-4ff3-b8db-b645ed1744cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.597414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d87f-4f2c-4ff3-b8db-b645ed1744cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.646875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:58.648367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:58.653970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3326d87f-4f2c-4ff3-b8db-b645ed1744cf None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:58.655851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3326d87f-4f2c-4ff3-b8db-b645ed1744cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:58.662110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 829/1697] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:58.673346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 869/1698] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:15:58.704653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:58.716102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71271-7a33-4439-ad61-a2dc7215706e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:58.720584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:58.721503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:58.721844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:58.722220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:58.730808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:58.732399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:58.742042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 830/1699] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:59.008746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.009466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.010098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.041109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] MFA Rules not processed for user `ed4711b7691643109007e173fee1c405`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:59.042445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:59.053392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 870/1700] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:59.308700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:59.327140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518b965-6d05-48b1-9b89-65612d68e7d8 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:59.362430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:59.363074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.363362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.363636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.381518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:59.383761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf638cb-b40b-448c-882a-333b954160f4 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:59.390171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:59.392411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:59.396046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 831/1701] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:15:59.457374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:15:59.503030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50df8-72b6-4362-abc0-00f5ddc208bb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.503316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50df8-72b6-4362-abc0-00f5ddc208bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.503593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50df8-72b6-4362-abc0-00f5ddc208bb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.507803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 832/1702] 10.222.0.40 () {54 vars in 961 bytes} [Sun Aug 28 10:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:15:59.516704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.521017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.521559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.531400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b928-4f83-474f-99b4-60525b982cdd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:59.546705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:59.547474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.547608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.548039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.550115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:59.562778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:15:59.564116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:15:59.565221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:59.566833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46731a76-5987-44be-820b-17fbd76d54d8 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:59.576847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 833/1703] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Aug 28 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:15:59.590508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:15:59.596212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 871/1704] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:15:59.613469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdb1f6-e10b-460a-a20f-a5e481686559 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:15:59.630632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:15:59.632525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:15:59.633032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:15:59.633524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:15:59.649254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:15:59.669909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9beef-98bd-4109-b4e1-3803f08530a7 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:15:59.719923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 834/1705] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:01.107547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:01.108845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:01.109292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:01.133521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] MFA Rules not processed for user `edbf26fde0254259a540f31139fc1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:01.135059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:01.149454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 872/1706] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:09.703936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=54 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:09.869416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:09.869701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:09.869978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:10.031747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:10.033164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:10.126603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 835/1707] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:10.262743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:10.265568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:10.414292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa0386-8eef-4a3c-b0f8-e24cfd8930e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:10.424902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:10.425543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:10.425820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:10.426400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:10.454293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:10.456685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917431f-8a77-4357-8102-744cba58ad2e service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:10.472110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 873/1708] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:16:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:16:20.121332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=40 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:20.464020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=61 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:26.496055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:26.499063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:26.575853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe57c4-4115-4a25-a837-0021ee500091 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:26.580806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:26.581465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:26.581747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:26.582076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] PATH_INFO: `/v3/users/807514ea5ca9442e986faf5318253bc8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:26.583290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=807514ea5ca9442e986faf5318253bc8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:26.585624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:26.645293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807514ea5ca9442e986faf5318253bc8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:26.652049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807514ea5ca9442e986faf5318253bc8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:26.652752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807514ea5ca9442e986faf5318253bc8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:26.662824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807514ea5ca9442e986faf5318253bc8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:26.663520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807514ea5ca9442e986faf5318253bc8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:26.680664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 836/1709] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:26 2022] DELETE /identity/v3/users/807514ea5ca9442e986faf5318253bc8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:26.685003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:26.685551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:26.685551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:26.704544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:26.705806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:26.769922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 874/1710] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:26.780052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:26.781629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:26.844825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a94ee-700b-487d-b893-b03eb77912ca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:26.848427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:26.849077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:26.849407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:26.849675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:26.859466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:26.861062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a86ba-72a6-4c67-9133-1b7dfd603311 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:26.872993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 837/1711] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.181977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.183456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.195544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde4bd7-b605-4d04-807e-f665196f10d3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.200067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.200885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.201001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.201270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] PATH_INFO: `/v3/projects/98de7ba1be02492b9a24eb3bd17c1192` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.202425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98de7ba1be02492b9a24eb3bd17c1192)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.204346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.256401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98de7ba1be02492b9a24eb3bd17c1192', 'request_id': 'req-d567521c-112e-432f-bf62-70de8731e900'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:27.265302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98de7ba1be02492b9a24eb3bd17c1192', 'request_id': 'req-d567521c-112e-432f-bf62-70de8731e900'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:27.272932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98de7ba1be02492b9a24eb3bd17c1192', 'request_id': 'req-d567521c-112e-432f-bf62-70de8731e900'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:27.284277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] The token cache is being invalidate because project 98de7ba1be02492b9a24eb3bd17c1192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:27.284597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:27.285268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d567521c-112e-432f-bf62-70de8731e900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:27.286916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 875/1712] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:27 2022] DELETE /identity/v3/projects/98de7ba1be02492b9a24eb3bd17c1192 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:27.292256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e2fc-6268-481f-b715-f06fa7ed74d8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.292651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e2fc-6268-481f-b715-f06fa7ed74d8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.292996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e2fc-6268-481f-b715-f06fa7ed74d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.309569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.309881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.310329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.314951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-48c4e2fc-6268-481f-b715-f06fa7ed74d8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:27.316164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48c4e2fc-6268-481f-b715-f06fa7ed74d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.331005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:27.332186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.334070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:27.376927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 838/1713] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:27.384428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.384747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.385020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.387357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 876/1714] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Aug 28 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:27.404469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:27.405746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.417077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 839/1715] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:27.422059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.430586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.436044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.437519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa0f9a-7168-4ac1-b0cc-985d40b61757 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.442156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.442896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.443311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.443677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.444433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:27.444887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.447580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.450647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e33ad6-e823-41b7-a9b3-934f76a2e017 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.455374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.456006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.456283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.456582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.468246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.469964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.470126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 877/1716] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:27.471210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.479622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 840/1717] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.480265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.481481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03802d4-eb85-40c0-9032-030030625524 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.481600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:27.485197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.485938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.486284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.486613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.493413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aac1d-e0e6-492a-8436-0e5876f460bc service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.494361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.495946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.497739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.498420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.499064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.500591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.505263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 878/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.506011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.508308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.509962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.514807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19ebfd-7ab3-4c9d-bba0-bbdddbdaa29e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.518553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.519171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.519439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.519708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.522442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 841/1719] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.523196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.524143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:27.533700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.535671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0267-2dcd-4d3c-9df6-e32d19053a17 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.536260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.541597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.542528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.543063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.543492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.546789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 879/1720] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.547675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.553016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.554761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.563283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff5c19-5b1d-4bb0-89aa-6c600d1ad3eb service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.567390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 842/1721] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:27.568764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.569385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.569661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.569992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.570628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.572584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.619146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 880/1722] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:27 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:27.623679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.637136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c586-75eb-43b2-9b5e-9fada1ff836a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.713913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.714647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.715138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.715504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.716400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.718521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.769081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 843/1723] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:27 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:27.770123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.790136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d3fb-1722-4031-9fad-9c2b3feb8d2d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.812378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.813112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.814111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.814458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.815772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.820778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.855862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 881/1724] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:16:27.859955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.876155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05f3b-feaf-45a8-8024-ff8174e29eb6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.887507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.888081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.888367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.888667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.891502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.896074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfed246-1721-4aa7-a79a-8e062fed6ea8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.928577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 844/1725] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:27.930849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:27.942446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1dc82-4732-4569-a5f4-dfa66655fcaf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:27.953019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:27.953705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:27.953978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:27.954244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] PATH_INFO: `/v3/projects/bf46992b8e874f5a8e206b2db4b160c6/users/b91f8db9479b4f7ab0cd1416fb6a020b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:27.971482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf46992b8e874f5a8e206b2db4b160c6, user_id=b91f8db9479b4f7ab0cd1416fb6a020b, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:27.974199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:27.992435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 882/1726] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:27 2022] PUT /identity/v3/projects/bf46992b8e874f5a8e206b2db4b160c6/users/b91f8db9479b4f7ab0cd1416fb6a020b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:28.006572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.006886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.007173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.047839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] MFA Rules not processed for user `b91f8db9479b4f7ab0cd1416fb6a020b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:28.049255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:28.127673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:28.164178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 845/1727] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:28.169070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.324850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e68d6-b03c-4bdd-b3c9-e028e5458c59 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.330606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.331406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.331751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.332214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.332965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.335365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.383666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 883/1728] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:28 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:28.387400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.398749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8690001-19e0-41e6-813f-6021a2e1e2fe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.470223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.470223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.470223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.470574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.471124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.473273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.501565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 846/1729] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:28 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:28.502791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.525813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d64d6-c8a7-428c-aec8-88505c23dfaa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.532443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.533488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.534668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.535081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.535951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.537913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.552744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 884/1730] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:28.557415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.569518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857545-8904-4acd-a394-132fcceb69a7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.574934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.575639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.576183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.576183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] PATH_INFO: `/v3/projects/db782bcf94f642e2b4cfe367943c0d5a/users/fb60802bb0ad4285b35c875666d4971f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.601274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=db782bcf94f642e2b4cfe367943c0d5a, user_id=fb60802bb0ad4285b35c875666d4971f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.604289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.623350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 847/1731] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:28 2022] PUT /identity/v3/projects/db782bcf94f642e2b4cfe367943c0d5a/users/fb60802bb0ad4285b35c875666d4971f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:28.627208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.637964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8964f1-9881-4e73-86a6-397b7800bee3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.708251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.709132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.709411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.709877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.710622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.711980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d3f3-b4ff-422e-bc4f-0adc58e7cf5f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.725356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 885/1732] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:28.730192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.745943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c36b28-9790-4e49-9e4f-a962dbdfc7ef admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:28.753874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:28.754798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.755158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.755696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] PATH_INFO: `/v3/projects/db782bcf94f642e2b4cfe367943c0d5a/users/fb60802bb0ad4285b35c875666d4971f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.758325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db782bcf94f642e2b4cfe367943c0d5a, user_id=fb60802bb0ad4285b35c875666d4971f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:28.761561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:28.774137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 848/1733] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:28 2022] PUT /identity/v3/projects/db782bcf94f642e2b4cfe367943c0d5a/users/fb60802bb0ad4285b35c875666d4971f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:28.788678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79076a2-a2bd-493b-8b69-2deadb1777e3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:28.790681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79076a2-a2bd-493b-8b69-2deadb1777e3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:28.791301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79076a2-a2bd-493b-8b69-2deadb1777e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:28.812070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:28.831975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e79076a2-a2bd-493b-8b69-2deadb1777e3 None None] MFA Rules not processed for user `fb60802bb0ad4285b35c875666d4971f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:28.833373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e79076a2-a2bd-493b-8b69-2deadb1777e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:28.931506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 886/1734] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:28.943501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:28.947437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:29.028743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ab65e-fdf3-433c-8d6a-9928516b692b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:29.032492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:29.033244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.033584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.033918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.044019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:29.045675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:29.057348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e95a-074e-4e8e-af29-6b7d7ca20d5c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.058466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e95a-074e-4e8e-af29-6b7d7ca20d5c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.059050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e95a-074e-4e8e-af29-6b7d7ca20d5c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.060739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 887/1735] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:16:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:16:29.064793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 849/1736] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:29.121484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.122144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.122602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.164366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:29.165610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:29.178760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:29.199971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf063d3-4efc-4636-a435-2ec2d1d4d38b service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:29.214075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:29.214372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.214624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.215448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.289848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 888/1737] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:29.310601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:29.350520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:29.352704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:29.369768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:29.370860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94600f-f5f3-403f-a48e-ac22d497ed41 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:29.371218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bbf6c-c39f-41c3-b3ee-1a00278b4ade service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:29.379455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:29.381473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.382123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 850/1738] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:29.388043 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.392308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.419475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:29.422107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253a098-dcad-47eb-aea7-c2eab4163c6a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:29.441417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 889/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:16:29.571661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baae1d-7008-4b7e-b4ed-9bf90b0bf08b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.572773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baae1d-7008-4b7e-b4ed-9bf90b0bf08b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.572863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baae1d-7008-4b7e-b4ed-9bf90b0bf08b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.574581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 851/1740] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:16:29.598739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5fa1c-ff89-48f5-8694-53c83e88e262 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.599136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5fa1c-ff89-48f5-8694-53c83e88e262 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.599505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5fa1c-ff89-48f5-8694-53c83e88e262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.630892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f5a5fa1c-ff89-48f5-8694-53c83e88e262 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:29.632110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5a5fa1c-ff89-48f5-8694-53c83e88e262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:29.643588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 890/1741] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:29.662959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baae1d-7008-4b7e-b4ed-9bf90b0bf08b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:29.681214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baae1d-7008-4b7e-b4ed-9bf90b0bf08b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:29.691347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:29.692022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:29.692489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:29.693045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:29.703992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:29.705963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ce6a-d471-4f54-b298-0dc761b917ca service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:29.718480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 852/1742] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:16:39.625984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=71 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:39.714199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=16 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:46.087181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28c4e0-7ffe-4703-b369-3777deb7a46c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:46.087690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28c4e0-7ffe-4703-b369-3777deb7a46c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:46.087690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28c4e0-7ffe-4703-b369-3777deb7a46c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:46.089369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 891/1743] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:16:46.096934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:46.097480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:46.097625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:46.122175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:46.123318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:46.133725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 853/1744] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:46.149901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28c4e0-7ffe-4703-b369-3777deb7a46c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:46.165861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28c4e0-7ffe-4703-b369-3777deb7a46c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:46.172231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:46.172878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:46.173419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:46.173728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:46.183803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:46.185478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa2af-a398-45d4-90d9-d28339b68f36 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:46.196575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 892/1745] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:46.724395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:46.726320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:46.804990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f245e-ebb6-4890-b4a3-5369d73a6f0f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:46.809994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:46.810768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:46.811051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:46.811371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] PATH_INFO: `/v3/users/555e8ced657b48fc8b6f38d00403008d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:46.812538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555e8ced657b48fc8b6f38d00403008d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:46.814608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:46.883332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555e8ced657b48fc8b6f38d00403008d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:46.893285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555e8ced657b48fc8b6f38d00403008d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:46.894043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555e8ced657b48fc8b6f38d00403008d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:46.908686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555e8ced657b48fc8b6f38d00403008d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:46.909503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555e8ced657b48fc8b6f38d00403008d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:46.933693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 854/1746] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:46 2022] DELETE /identity/v3/users/555e8ced657b48fc8b6f38d00403008d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:46.938422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:46.939006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:46.939961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:46.971077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:46.972392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.037320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 893/1747] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:47.047649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:47.049093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.107904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff2219-c307-46a1-9d86-57aa3843a7a8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:47.111724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:47.112394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.112699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.112978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.122700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:47.124139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:47.133728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 855/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:16:47.622040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:47.623504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:47.623956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.625072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.640309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5ec7f-302c-49ab-89ed-f9dc058baffe None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:47.645645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629919-4b69-4ef4-be21-8b823c5f5bec service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:47.648866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:47.649461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.650023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.650426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] PATH_INFO: `/v3/users/c8d1a4630510480ab8deb3f30ffbb83c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.652440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d1a4630510480ab8deb3f30ffbb83c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:47.653378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:47.654034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.654309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.654578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] PATH_INFO: `/v3/projects/d88cb2f7d9d243eab09d20334b1e0044` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.656049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88cb2f7d9d243eab09d20334b1e0044)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:47.656312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:47.658591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:47.689977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:47.747163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d1a4630510480ab8deb3f30ffbb83c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.755513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d1a4630510480ab8deb3f30ffbb83c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.765308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88cb2f7d9d243eab09d20334b1e0044', 'request_id': 'req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.777425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88cb2f7d9d243eab09d20334b1e0044', 'request_id': 'req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.785333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88cb2f7d9d243eab09d20334b1e0044', 'request_id': 'req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.798424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d1a4630510480ab8deb3f30ffbb83c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.803985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] The token cache is being invalidate because project d88cb2f7d9d243eab09d20334b1e0044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:47.804283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.804971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.806611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 856/1749] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:47 2022] DELETE /identity/v3/projects/d88cb2f7d9d243eab09d20334b1e0044 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:47.808119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d1a4630510480ab8deb3f30ffbb83c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.808854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-427e1d02-6ebd-4826-bb44-6f70776ee171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d1a4630510480ab8deb3f30ffbb83c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:47.811006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:47.812966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.819586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 894/1750] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:47 2022] DELETE /identity/v3/users/c8d1a4630510480ab8deb3f30ffbb83c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:47.821636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.821924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.822200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.844766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:47.846233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.898213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1de1f4-d017-4d9d-8e56-57c3b5c026ec admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:47.903588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:47.920301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 895/1751] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:47.921436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:47.923003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:47.935531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e86fc-54b5-4d51-b49d-7cad97a2c0b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:47.941945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:47.942662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.942950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.943224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] PATH_INFO: `/v3/users/2cb6d63eadc14cff8515bc3ed8da99bc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.944372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb6d63eadc14cff8515bc3ed8da99bc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:47.946414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:47.967706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:47.968430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:47.968806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:47.969148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] PATH_INFO: `/v3/users/5a930fd048c4452ab4a02d089762b99a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:47.970520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a930fd048c4452ab4a02d089762b99a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:47.974346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.012260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb6d63eadc14cff8515bc3ed8da99bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.021651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb6d63eadc14cff8515bc3ed8da99bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.042184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb6d63eadc14cff8515bc3ed8da99bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.059593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb6d63eadc14cff8515bc3ed8da99bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.060427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb6d63eadc14cff8515bc3ed8da99bc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.073151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 896/1752] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:47 2022] DELETE /identity/v3/users/2cb6d63eadc14cff8515bc3ed8da99bc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:48.074206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.078662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.080811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a930fd048c4452ab4a02d089762b99a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.082821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:48.095565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a930fd048c4452ab4a02d089762b99a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.098116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a930fd048c4452ab4a02d089762b99a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.115945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a930fd048c4452ab4a02d089762b99a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.116705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f017c80d-ce9a-42c9-98b0-817138ba9c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a930fd048c4452ab4a02d089762b99a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.141761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 857/1753] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:47 2022] DELETE /identity/v3/users/5a930fd048c4452ab4a02d089762b99a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:48.144351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.145857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.146250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.173950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:48.176104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.227722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0b93f-39f2-42ef-bbc3-f2a91530bea2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.277270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 858/1754] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:48.277270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.277270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.292938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4a73-9d88-4c17-984e-0109b02cf159 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.296270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.297274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.297752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.298166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.299799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.300573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.300971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.301206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.301778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] PATH_INFO: `/v3/users/b91f8db9479b4f7ab0cd1416fb6a020b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.303175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91f8db9479b4f7ab0cd1416fb6a020b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.305599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.317819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.320033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.340246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 897/1755] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:48.342470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.344228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.359501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170c3a1-046b-4ff6-9124-63e81b29ab71 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.378750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91f8db9479b4f7ab0cd1416fb6a020b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.382288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.383838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.385297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.385907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.397914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91f8db9479b4f7ab0cd1416fb6a020b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.399685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91f8db9479b4f7ab0cd1416fb6a020b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.406135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.407741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383a132-2757-4ca6-a985-5e63813f9fc5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.422724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91f8db9479b4f7ab0cd1416fb6a020b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.423413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91f8db9479b4f7ab0cd1416fb6a020b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.426557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 898/1756] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:48.452687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 859/1757] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:48 2022] DELETE /identity/v3/users/b91f8db9479b4f7ab0cd1416fb6a020b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:48.454903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.455501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.455785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.477282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:48.478511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.492770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:48.565335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 899/1758] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:48.585265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.588387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.602946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.604900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.632059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbed89e-2bbb-4010-bbb6-7a771fa19728 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.639564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.640809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.641184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.641595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] PATH_INFO: `/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.644041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c9a9b5ff2445d4b85bc584d2b0602a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.646721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.698395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:48.720463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c9a9b5ff2445d4b85bc584d2b0602a', 'request_id': 'req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.734744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c9a9b5ff2445d4b85bc584d2b0602a', 'request_id': 'req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.738870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189edb7-385b-4c7d-8c27-4c180163f497 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.746204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c9a9b5ff2445d4b85bc584d2b0602a', 'request_id': 'req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.763421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] The token cache is being invalidate because project 47c9a9b5ff2445d4b85bc584d2b0602a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:48.763719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.764447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-489f6ac2-e7f4-448b-8ea9-9bbd0fba3df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:48.766197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 900/1759] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:48 2022] DELETE /identity/v3/projects/47c9a9b5ff2445d4b85bc584d2b0602a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:48.771968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.772311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.772635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.812879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:48.814560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.851328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.852261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.853040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.854141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.856949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.956283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 901/1760] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:48.957168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.957299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.958457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.958918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.969523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 860/1761] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:48.970396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:48.971678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:48.971817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16774cd-ee06-4cc0-ac57-5f7f78402cf8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.978420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.979520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.979520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.979884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] PATH_INFO: `/v3/projects/2df4ce3a5f7e4dbb95d16e3f7bdae142` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.981141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df4ce3a5f7e4dbb95d16e3f7bdae142)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.983171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:48.984034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c93b-a620-4e10-b8ca-8af92a59c0a0 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:48.989040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:48.989872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:48.990373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:48.990651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] PATH_INFO: `/v3/projects/a7c28643a59f479b8ebda21d3d2c1df5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:48.991823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c28643a59f479b8ebda21d3d2c1df5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:48.993843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.027622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.039191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df4ce3a5f7e4dbb95d16e3f7bdae142', 'request_id': 'req-459e5be3-005e-426b-886d-c34f094069b5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.049375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df4ce3a5f7e4dbb95d16e3f7bdae142', 'request_id': 'req-459e5be3-005e-426b-886d-c34f094069b5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.054085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c28643a59f479b8ebda21d3d2c1df5', 'request_id': 'req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.059317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df4ce3a5f7e4dbb95d16e3f7bdae142', 'request_id': 'req-459e5be3-005e-426b-886d-c34f094069b5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.066712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c28643a59f479b8ebda21d3d2c1df5', 'request_id': 'req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.073151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c28643a59f479b8ebda21d3d2c1df5', 'request_id': 'req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.073554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] The token cache is being invalidate because project 2df4ce3a5f7e4dbb95d16e3f7bdae142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:49.074233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.074860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-459e5be3-005e-426b-886d-c34f094069b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.076890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 902/1762] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:48 2022] DELETE /identity/v3/projects/2df4ce3a5f7e4dbb95d16e3f7bdae142 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:49.078155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b20e2-b415-49e8-9124-bc7dae56a03d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.078526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b20e2-b415-49e8-9124-bc7dae56a03d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.078869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b20e2-b415-49e8-9124-bc7dae56a03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.090416 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] The token cache is being invalidate because project a7c28643a59f479b8ebda21d3d2c1df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:49.090714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.091394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.096325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 861/1763] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:48 2022] DELETE /identity/v3/projects/a7c28643a59f479b8ebda21d3d2c1df5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:49.096325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.096325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.103604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c73b20e2-b415-49e8-9124-bc7dae56a03d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:49.105176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c73b20e2-b415-49e8-9124-bc7dae56a03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.130330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.180555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7f6aa-e350-4c60-83b4-3dc5f141d257 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.185433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.186130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.186354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.186631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] PATH_INFO: `/v3/users/d6427967e2234a189fdcd44a7f802942` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.187797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6427967e2234a189fdcd44a7f802942)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.189776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.192455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 903/1764] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:49.194265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.194720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.195236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.219860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:49.221240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.232663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 904/1765] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:49.233701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.244640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b30d0a-734d-43c8-abd1-b623ff85c4ce None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.249638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.250317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.250879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.251084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.251816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:49.252171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.253348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6427967e2234a189fdcd44a7f802942'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.253954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8802a-2b4d-4026-8809-66dc0ad79e27 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.262787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6427967e2234a189fdcd44a7f802942'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.263805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6427967e2234a189fdcd44a7f802942'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.271468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 905/1766] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:49.271468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.271468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.271468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.278206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6427967e2234a189fdcd44a7f802942'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.278909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6427967e2234a189fdcd44a7f802942'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.293745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:49.294953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.301758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 862/1767] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:49 2022] DELETE /identity/v3/users/d6427967e2234a189fdcd44a7f802942 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:49.302524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.304039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.305236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 906/1768] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:49.310792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.326070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e46401-a37c-44d9-992c-f7431180ece4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.385348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c1c4-772f-4a20-bbd5-04ccebf3f25f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.394170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.394896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.395936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.396228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.397489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.400356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.430423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.431414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.431794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.431994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.444829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:49.445740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.447593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.472061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 907/1769] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:49.474033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 863/1770] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/projects => generated 399 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:49.475378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.485089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.488030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.502660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1f3e-10d2-4749-bb23-5ea7d1967b8d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.531299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.543076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.596759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.597588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1e33-aecd-49d6-a836-8b0a33d48816 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.597880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.598325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.598717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] PATH_INFO: `/v3/projects/bf46992b8e874f5a8e206b2db4b160c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.600172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf46992b8e874f5a8e206b2db4b160c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.602483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.606039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.608245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.608685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.609383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.610156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.612731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.645616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 864/1771] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:49.646908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.660722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36f240-c68d-4cd4-b2cc-554ccb4c664d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.677679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf46992b8e874f5a8e206b2db4b160c6', 'request_id': 'req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.698321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf46992b8e874f5a8e206b2db4b160c6', 'request_id': 'req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.716079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf46992b8e874f5a8e206b2db4b160c6', 'request_id': 'req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.732047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] The token cache is being invalidate because project bf46992b8e874f5a8e206b2db4b160c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:49.732253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.732970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.738255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 908/1772] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:49 2022] DELETE /identity/v3/projects/bf46992b8e874f5a8e206b2db4b160c6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:49.739396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.740874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.753438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.754242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.754403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.754604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.755262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.757265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.795099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 865/1773] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:49 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:49.795099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.797730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.820107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad82a8-d204-4440-80f9-2cc5c2f116ee admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.880450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c5d8-a311-4d4e-9e24-7520ac3300e5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.898700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.898700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.898700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.898700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] PATH_INFO: `/v3/projects/621c25f156754c9593ca7d67512b28a0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.899118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=621c25f156754c9593ca7d67512b28a0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.902624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.902800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:49.903609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:49.904096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:49.904663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:49.905595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:49.908035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:49.932529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 909/1774] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:49.932529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:49.937990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:49.952385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef048b3b-6f95-4f1a-a8b1-6e259af7b5a9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:49.957715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.962427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:49.964840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621c25f156754c9593ca7d67512b28a0', 'request_id': 'req-a2655c02-3682-4101-a2a4-77b604086ae9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.977196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621c25f156754c9593ca7d67512b28a0', 'request_id': 'req-a2655c02-3682-4101-a2a4-77b604086ae9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:49.985576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621c25f156754c9593ca7d67512b28a0', 'request_id': 'req-a2655c02-3682-4101-a2a4-77b604086ae9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.002570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] The token cache is being invalidate because project 621c25f156754c9593ca7d67512b28a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:50.002861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.003527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.005931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 866/1775] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:49 2022] DELETE /identity/v3/projects/621c25f156754c9593ca7d67512b28a0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:50.006960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.008434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.023094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.024413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.025060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.025506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] PATH_INFO: `/v3/users/fb60802bb0ad4285b35c875666d4971f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.026961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb60802bb0ad4285b35c875666d4971f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.030792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.084272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb60802bb0ad4285b35c875666d4971f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.084659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655c02-3682-4101-a2a4-77b604086ae9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.090054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.090766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.091245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.091636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.092440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.093225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb60802bb0ad4285b35c875666d4971f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.094480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.108181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb60802bb0ad4285b35c875666d4971f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.118053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb60802bb0ad4285b35c875666d4971f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.118904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-832d5944-b708-4677-ab8d-0bcdbfb5dead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb60802bb0ad4285b35c875666d4971f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.124238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 867/1776] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:50.124238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.124238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.126113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 910/1777] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:49 2022] DELETE /identity/v3/users/fb60802bb0ad4285b35c875666d4971f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:50.127404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.127689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.128076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.141199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e264ae7-ff13-484d-b85f-2a749010db85 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.149318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:50.150553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.236600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 911/1778] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:50.237828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.242502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.266854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df41c71-20ef-48d7-931b-4fb087b42f46 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.271894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.272542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.272823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.274890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.276978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.280135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7aa63-2459-4528-b308-2b4d973313d9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.281254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.282417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.282659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.282938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.283714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.285275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.301985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 868/1779] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:50.304618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.305066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.306250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.319104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 912/1780] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:50.319697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.347833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ee49-9805-4453-8acf-b20d63afef5e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.354348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:50.357473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.363458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.363458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.363458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.363458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] PATH_INFO: `/v3/projects/61b8e6df3a7b460280566118e1b55701/users/a2a1c5ec891a41ed9050e87cb58021ca/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.378267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 869/1781] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:50.379489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.381174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:50.384698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b8e6df3a7b460280566118e1b55701, user_id=a2a1c5ec891a41ed9050e87cb58021ca, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.387817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.398193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee8535-626a-40f5-a544-33747fd0e295 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.410306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 913/1782] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:50 2022] PUT /identity/v3/projects/61b8e6df3a7b460280566118e1b55701/users/a2a1c5ec891a41ed9050e87cb58021ca/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:50.411582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.504822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:50.508340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b782d-b369-4484-a036-42367a19578c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.513968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.514132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.514406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.514677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.515386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:50.515564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.515856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.516444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.517363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.517528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae1f5-a31b-4c79-9774-b76456291c2a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.518419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.519322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.521493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.531633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 914/1783] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:50.533132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.533585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.533915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 870/1784] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:50.534162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.537711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.548813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122a990-fc16-435f-a37b-19235c492c5e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.554187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.554887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.555241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.555566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.556345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.558687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.582106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] MFA Rules not processed for user `a2a1c5ec891a41ed9050e87cb58021ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:50.583560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.603020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 871/1785] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:50.604135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.676007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 915/1786] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:50.677224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.683613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319a54b-ca58-4764-82d0-201a79f1eb7e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.689329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.690097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.690520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.690865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] PATH_INFO: `/v3/projects/f03e3971600e4c83913b69a86392402d/users/6c5e927518d44e84b6cd187fdfeaa558/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.691524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05640342-ac51-48dc-8087-1d571037ce0b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.697093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.697822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.698046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.698390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.699164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.701772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.709221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03e3971600e4c83913b69a86392402d, user_id=6c5e927518d44e84b6cd187fdfeaa558, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.714751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.730077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 916/1787] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/users => generated 283 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:50.734095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.740776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 872/1788] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:50 2022] PUT /identity/v3/projects/f03e3971600e4c83913b69a86392402d/users/6c5e927518d44e84b6cd187fdfeaa558/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:50.749945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.749945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:50.750136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b03c0-a2a9-4788-a3fc-5befb1949f25 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.801609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:50.809734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.810407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.810698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.811275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.812353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.814599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811a5e2-400a-4c4d-89c3-0edb45b02f0b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:50.815017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.821342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:50.822163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.822605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.823007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] PATH_INFO: `/v3/projects/db782bcf94f642e2b4cfe367943c0d5a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:50.824350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db782bcf94f642e2b4cfe367943c0d5a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:50.826566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:50.866529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 917/1789] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:50.866529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:50.891984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db782bcf94f642e2b4cfe367943c0d5a', 'request_id': 'req-d5428866-2141-4db6-9f0e-b23ee2be9b52'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.897118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:50.899557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db782bcf94f642e2b4cfe367943c0d5a', 'request_id': 'req-d5428866-2141-4db6-9f0e-b23ee2be9b52'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.915001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db782bcf94f642e2b4cfe367943c0d5a', 'request_id': 'req-d5428866-2141-4db6-9f0e-b23ee2be9b52'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.948055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] The token cache is being invalidate because project db782bcf94f642e2b4cfe367943c0d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:50.949338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.950237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5428866-2141-4db6-9f0e-b23ee2be9b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:50.954278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 873/1790] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:50 2022] DELETE /identity/v3/projects/db782bcf94f642e2b4cfe367943c0d5a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:50.955837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:50.956383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:50.958020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.047917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] MFA Rules not processed for user `6c5e927518d44e84b6cd187fdfeaa558`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:51.053851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.062089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234768-6a88-4d40-9378-3d0bcc2e8da8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.202058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 874/1791] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:51.203453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.204189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.204303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.204698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.205471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.206346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.206790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.208256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece21ac5-b93c-413d-8a40-0176733ddcc5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.223869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 918/1792] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:51.223869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.223869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.224946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.227384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75295d-fbb5-4eb3-a88d-b24cd93dab5a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.232756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.233581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.233997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.234724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.235505 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.237663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.249903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:51.251176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.263979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 919/1793] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:51.271855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.281248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 875/1794] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/users => generated 307 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:51.282547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.283205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.289632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.308794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b54e0b-1d06-4386-ba1a-2f301fc65782 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.317864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.318848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021c2d4-181f-41d2-a596-f70679b9b073 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.319365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.319828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.320347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.321332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.323308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57384eed-d9c1-43b5-8471-e938667997db admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.325493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.326413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.326736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.327113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.328236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.330658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.346147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 876/1795] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:51.348147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.348602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.349007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.373361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 920/1796] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:51.374666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.378772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:51.380236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.491379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:51.496250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094562-e340-44bc-9d58-754d546a6867 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.506502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.507437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.507817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.508141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.508954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.511005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.519215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 877/1797] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:51.520481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.531668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 921/1798] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:51.532585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.539938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48049500-61bb-4417-9d11-6bc401a31e0d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.546832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.547630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.548380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.549465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] PATH_INFO: `/v3/projects/61afb8edc9bf4f008c6d7a363350af02/users/03300cd9e4fa418583263336f15b8148/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.556180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a310e-57d4-40f4-a6c1-a42dbdbf8cfc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.568721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.569348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.569671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.569958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.571240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.573809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.575020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=61afb8edc9bf4f008c6d7a363350af02, user_id=03300cd9e4fa418583263336f15b8148, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.578007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.612659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 878/1799] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:51 2022] PUT /identity/v3/projects/61afb8edc9bf4f008c6d7a363350af02/users/03300cd9e4fa418583263336f15b8148/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:51.612659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.633088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 922/1800] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:51.634857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.639101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:51.649588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0eaf1-321d-49ef-a24b-a6a07ca0f00b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.688940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec40687-5dd3-4c3c-a0be-4d023af2a5cc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.694883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.695565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.695926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.696375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.697140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:51.697509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.699262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83971b8-9c4b-4150-aa6a-6c9aba559479 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.714483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 879/1801] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:51.719272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.719557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.719827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.743873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.744571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.744850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.745138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] PATH_INFO: `/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7/users/814cb08c186245f685831c490639a1dd/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.754965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] MFA Rules not processed for user `03300cd9e4fa418583263336f15b8148`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:51.756238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:51.765379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf0dab7b752409bbeb405236e3c2bd7, user_id=814cb08c186245f685831c490639a1dd, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.769720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.791840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 923/1802] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:51 2022] PUT /identity/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7/users/814cb08c186245f685831c490639a1dd/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:51.792942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.839747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 880/1803] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:51.839747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.852420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76385b-aac0-4912-ad94-99033fe692d1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.858179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.859120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.859714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.860153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.860841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.862706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7049-e6dc-4eb6-899d-6e168b94b6eb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.862817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.871100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.871822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.872102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.872369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.873150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.874755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.885461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 924/1804] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:51.889299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.891945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 881/1805] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:51.892790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.898811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003733a-8492-471c-aa4b-9c67dd635aa1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.947824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:51.952285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9b407-c334-4cde-bbe7-815a83d1a76e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.955878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.956803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.957075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.957196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.957329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.957918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.958023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.958223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.958494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.959126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.959575 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.961036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.969612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 925/1806] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:51.973652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.983406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bedac5-b148-4e82-a317-c5204f74efdd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:51.988323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:51.988829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 882/1807] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:51.988962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:51.989240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:51.989511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:51.989646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:51.990316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:51.991879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:51.998155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:52.006035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 926/1808] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:52.006035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.014219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8f097-10cf-412f-82fa-b7a7d0ce43aa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.056907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579479-88c5-415c-a40a-7b38162c70b0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.061666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.062276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.062553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.062823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.063450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.065399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.068311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.069003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.069277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.069548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] PATH_INFO: `/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7/users/814cb08c186245f685831c490639a1dd/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.071414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf0dab7b752409bbeb405236e3c2bd7, user_id=814cb08c186245f685831c490639a1dd, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.074094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.084677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 927/1809] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:52 2022] PUT /identity/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7/users/814cb08c186245f685831c490639a1dd/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:52.094229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 883/1810] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:51 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:52.094229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.094446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.103835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ed8a-e475-4547-af90-59eabae6acac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.158289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5d18f-9403-4569-989e-2d27627c2ac4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.161401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.162023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.162305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.162577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.163219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.164314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.165006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.165287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.165546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.165637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] PATH_INFO: `/v3/projects/9591ce42c1e84404a83655d8eb730c60/users/ea8aac4d2951482fafdde4dbf9100ae0/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.181006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9591ce42c1e84404a83655d8eb730c60, user_id=ea8aac4d2951482fafdde4dbf9100ae0, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.184293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d28d6-8339-4a9f-bf0b-afbf9abce6ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.191985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 884/1811] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:52.192873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.201540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 928/1812] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:52 2022] PUT /identity/v3/projects/9591ce42c1e84404a83655d8eb730c60/users/ea8aac4d2951482fafdde4dbf9100ae0/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:52.203004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23504-aa4c-408d-a527-575caaa2aaee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.203119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207daf1-8d46-4b86-a1d5-8548b5509668 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.203547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23504-aa4c-408d-a527-575caaa2aaee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.204062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f23504-aa4c-408d-a527-575caaa2aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.241374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-33f23504-aa4c-408d-a527-575caaa2aaee None None] MFA Rules not processed for user `814cb08c186245f685831c490639a1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:52.242854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33f23504-aa4c-408d-a527-575caaa2aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:52.271133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.271894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.272233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.272908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.273580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.275249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.291612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 885/1813] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:52.291612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.294390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:52.303299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be16fe-7708-4ee3-8b9b-d65c35626870 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.308436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.309114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.309392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.309659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.310309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.312029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.323732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 929/1814] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:52.324429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 886/1815] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:52.325518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.330072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d811a-613d-469a-818e-22f335d4559e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.331308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d811a-613d-469a-818e-22f335d4559e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.331810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d811a-613d-469a-818e-22f335d4559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.336544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ed2c-70ee-4dcf-bc90-3739a2dab98f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.341906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.342538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.342825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.343247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.343907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.345558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.351641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:52.358104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 887/1816] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:52.361504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.373389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce975-443c-4fee-8b41-6df87a1bd4d9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.377009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a88d811a-613d-469a-818e-22f335d4559e None None] MFA Rules not processed for user `ea8aac4d2951482fafdde4dbf9100ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:52.378645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a88d811a-613d-469a-818e-22f335d4559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:52.379444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.380132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.380389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.380780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] PATH_INFO: `/v3/projects/52a43d9081104fb39f8da33517ee0965/users/42c3dcd2154a48e186218750bc335e10/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.397445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a43d9081104fb39f8da33517ee0965, user_id=42c3dcd2154a48e186218750bc335e10, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.400455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.420250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 888/1817] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:52 2022] PUT /identity/v3/projects/52a43d9081104fb39f8da33517ee0965/users/42c3dcd2154a48e186218750bc335e10/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:52.420250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.420250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:52.460124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 930/1818] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:52.464189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d811a-613d-469a-818e-22f335d4559e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.475183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d811a-613d-469a-818e-22f335d4559e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.486399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f6b39-e8dd-4d5e-b7f1-022e2f068670 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.490149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.490773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.491051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.491321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.563624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.563835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.564836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.565293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.565836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] PATH_INFO: `/v3/projects/3310c7884a7b4394bb8de893461441fe/users/bf6d3ef10dd4463885d184b13c9c1022/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.567003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.578691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 889/1819] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:52.579632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.601175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3310c7884a7b4394bb8de893461441fe, user_id=bf6d3ef10dd4463885d184b13c9c1022, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.605232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.606052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1cb6-74ff-4c81-a3d2-30275c6ce3ce service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.612330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.613229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.613710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.614045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.614706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.616622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b97d7-5821-4150-accb-0e688ccd72e8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.626392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 931/1820] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:52 2022] PUT /identity/v3/projects/3310c7884a7b4394bb8de893461441fe/users/bf6d3ef10dd4463885d184b13c9c1022/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:52.627624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.629153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 890/1821] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:52.637746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.638125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.638834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.698315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] MFA Rules not processed for user `bf6d3ef10dd4463885d184b13c9c1022`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:52.699715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:52.775400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:52.784256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d095e96-4262-4a70-aa18-f7889593cf45 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.797007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.797639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.798103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.798375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.799021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.802544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.812346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 891/1822] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:52.813519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.825770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:52.836320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5070ef-1107-4fc2-9442-e706429a81d0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:52.844497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:52.846904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:52.847579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:52.848185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] PATH_INFO: `/v3/projects/52a43d9081104fb39f8da33517ee0965/users/42c3dcd2154a48e186218750bc335e10/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:52.851772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a43d9081104fb39f8da33517ee0965, user_id=42c3dcd2154a48e186218750bc335e10, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:52.855595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:52.862910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 932/1823] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:52.864311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.869699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 892/1824] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:52 2022] PUT /identity/v3/projects/52a43d9081104fb39f8da33517ee0965/users/42c3dcd2154a48e186218750bc335e10/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:52.878925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:52.966266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7b01-eb8f-450e-b0cb-3870d5ee3313 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.001512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5b465-ce79-412e-bb60-1d411351e9a7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.006764 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.007387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.007678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.008058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.008743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.011305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.039676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.040816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.041215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.041659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.046137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:53.067462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 893/1825] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:52 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:53.068394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.117852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.119888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a207-0c94-42b2-b071-43e9ea09c59f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.140776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 933/1826] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:53.144903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.145185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.145451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.156449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4840-d58c-4aba-a73b-a7069d70bf89 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.161212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.161956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.162300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.162636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.164183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.167174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.210923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] MFA Rules not processed for user `42c3dcd2154a48e186218750bc335e10`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:53.212487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:53.216349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 894/1827] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:53 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:53.219231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.242692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb248-21d4-427e-aecf-7dc43146bede admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.260758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.262178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.262178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.262474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.263062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.265235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.268783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:53.307386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 934/1828] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:53.308666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 895/1829] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:53 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:53.309048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.313729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.325908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa30c0-3e16-4a6a-9175-4617be37ef3d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.328873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe15edf-eb78-4b70-9b61-d16cbe61b93c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.334319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.335064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.335385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.335696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.336400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.338250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.338888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.339587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.339938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.340151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.340896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.344179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.354355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 935/1830] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:53.355470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.358032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:53.362248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 896/1831] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:53.362248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.376795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733baefc-d8ae-497c-affa-c1a1c80e5df3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.382118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.382804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.383297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.383497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] PATH_INFO: `/v3/projects/05143cab443f4cfcb297c4d77e1629be/users/f11ff4b1c6e5477bb30800ec86bcf43a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.406335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05143cab443f4cfcb297c4d77e1629be, user_id=f11ff4b1c6e5477bb30800ec86bcf43a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.409344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.425963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 897/1832] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:53 2022] PUT /identity/v3/projects/05143cab443f4cfcb297c4d77e1629be/users/f11ff4b1c6e5477bb30800ec86bcf43a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:53.426816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.439440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8ff-4644-40af-8320-82db7c02f861 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.441525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:53.501040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002dca1-de57-4c7e-8401-cc2240cba5c2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.506356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.507350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.507632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.508080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] PATH_INFO: `/v3/projects/9ac9baf812524c6da028985f9ab8fff7/users/5a2c1ea6f60246ef94f2846a3cd42f7d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.513167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.514682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.515008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.515366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.537812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac9baf812524c6da028985f9ab8fff7, user_id=5a2c1ea6f60246ef94f2846a3cd42f7d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.541101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.558881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 898/1833] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:53 2022] PUT /identity/v3/projects/9ac9baf812524c6da028985f9ab8fff7/users/5a2c1ea6f60246ef94f2846a3cd42f7d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:53.561411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.592463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.593942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.603876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 936/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:53.604922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.615016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091498-a2fc-44fb-af30-ca46ecdd2418 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.628883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc36e68-2a66-4088-a118-a80fc5b4ff00 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.634026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.635012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.635012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.635405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.636156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.637855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.648836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:53.656729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 899/1835] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:53.657600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.670105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf4fd3-fbe1-488b-9095-4089f485026e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.675232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.676003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.676385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.676799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] PATH_INFO: `/v3/projects/05143cab443f4cfcb297c4d77e1629be/users/f11ff4b1c6e5477bb30800ec86bcf43a/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.684438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05143cab443f4cfcb297c4d77e1629be, user_id=f11ff4b1c6e5477bb30800ec86bcf43a, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.684438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.684642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.684642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.684642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.684642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.684956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.686640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3956b4-f19e-45e9-91c8-e34b9da25ac8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.693460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 900/1836] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:53 2022] PUT /identity/v3/projects/05143cab443f4cfcb297c4d77e1629be/users/f11ff4b1c6e5477bb30800ec86bcf43a/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:53.694628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.701304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 937/1837] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:53.704564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.704997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.705272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.733773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] MFA Rules not processed for user `f11ff4b1c6e5477bb30800ec86bcf43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:53.734991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:53.749596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09911de-4178-4958-9993-6ca5d931bc45 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.753256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.753876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.754157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.754436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.789460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:53.803100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 938/1838] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:53.804022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.820111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.821714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.830700 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 901/1839] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:53.831480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:53.839354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adb955-9782-45e0-a68a-95fd01ceecfe service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.843436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.844665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.844947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.845223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.868841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c20867-a532-47c3-8a0f-5984d3819ba0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:53.875000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:53.875679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.875948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.876219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] PATH_INFO: `/v3/projects/9ac9baf812524c6da028985f9ab8fff7/users/5a2c1ea6f60246ef94f2846a3cd42f7d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.878161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac9baf812524c6da028985f9ab8fff7, user_id=5a2c1ea6f60246ef94f2846a3cd42f7d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.880879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56eb9d4-04d2-4e8d-9037-e14caca705b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.894054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 939/1840] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:53 2022] PUT /identity/v3/projects/9ac9baf812524c6da028985f9ab8fff7/users/5a2c1ea6f60246ef94f2846a3cd42f7d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:53.907857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092871-4755-4d99-828d-15303627cbb2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:53.908252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092871-4755-4d99-828d-15303627cbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:53.908557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092871-4755-4d99-828d-15303627cbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:53.915081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:53.916828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:53.930506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 902/1841] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:53.953004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-23092871-4755-4d99-828d-15303627cbb2 None None] MFA Rules not processed for user `5a2c1ea6f60246ef94f2846a3cd42f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:53.954414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-23092871-4755-4d99-828d-15303627cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:54.030012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:54.072603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 940/1842] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:54.083959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:54.093310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4548-f6db-43bf-8da2-8326c991e6f6 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:54.153992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:54.155048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:54.155513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:54.155803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:54.184533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:54.184533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03ad09-113c-4f52-b4c7-fda495ee2be8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:54.189755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 903/1843] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:54.229939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092871-4755-4d99-828d-15303627cbb2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:54.351417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092871-4755-4d99-828d-15303627cbb2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:54.357487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:54.358155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:54.358675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:54.359089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] PATH_INFO: `/v3/users/bf6d3ef10dd4463885d184b13c9c1022` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:54.360787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6d3ef10dd4463885d184b13c9c1022)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:54.362918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:54.461851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6d3ef10dd4463885d184b13c9c1022'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:54.485833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6d3ef10dd4463885d184b13c9c1022'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:54.498171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6d3ef10dd4463885d184b13c9c1022'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:54.509668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6d3ef10dd4463885d184b13c9c1022'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:54.510371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6d3ef10dd4463885d184b13c9c1022'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:54.517478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 941/1844] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:54 2022] DELETE /identity/v3/users/bf6d3ef10dd4463885d184b13c9c1022 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:54.521813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:54.522303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:54.522564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:54.547624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:54.549009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:54.638766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 904/1845] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:54.648607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:54.650165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:54.728751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97c3e6-bcc0-4167-869f-a24babe00f67 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:54.732371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:54.733035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:54.733313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:54.733580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:54.739622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:54.743635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:54.745387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:54.756484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 942/1846] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:55.068042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:55.069605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:55.084551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1819c-18cc-42ab-bd35-dcfd529248d6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:55.103983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:55.104671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:55.104951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:55.105314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] PATH_INFO: `/v3/projects/3310c7884a7b4394bb8de893461441fe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:55.106440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3310c7884a7b4394bb8de893461441fe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:55.108498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:55.121987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:55.164991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3310c7884a7b4394bb8de893461441fe', 'request_id': 'req-b8577407-d745-4e25-8110-b238a5c00cb7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.170570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3310c7884a7b4394bb8de893461441fe', 'request_id': 'req-b8577407-d745-4e25-8110-b238a5c00cb7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.177188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3310c7884a7b4394bb8de893461441fe', 'request_id': 'req-b8577407-d745-4e25-8110-b238a5c00cb7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.190605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] The token cache is being invalidate because project 3310c7884a7b4394bb8de893461441fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:55.190894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.191528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.196012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 905/1847] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:55 2022] DELETE /identity/v3/projects/3310c7884a7b4394bb8de893461441fe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:55.197261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:55.216259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486d052-1a08-4e95-bc68-a1891f8ce2a5 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:55.304314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:55.305496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:55.306007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:55.306487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] PATH_INFO: `/v3/users/5a2c1ea6f60246ef94f2846a3cd42f7d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:55.308061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2c1ea6f60246ef94f2846a3cd42f7d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:55.310170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:55.387930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2c1ea6f60246ef94f2846a3cd42f7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.399433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2c1ea6f60246ef94f2846a3cd42f7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.427808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2c1ea6f60246ef94f2846a3cd42f7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.441302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2c1ea6f60246ef94f2846a3cd42f7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.442703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2c1ea6f60246ef94f2846a3cd42f7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:55.458984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 943/1848] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:55 2022] DELETE /identity/v3/users/5a2c1ea6f60246ef94f2846a3cd42f7d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:55.667577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:55.670447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:55.803285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577407-d745-4e25-8110-b238a5c00cb7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:55.881253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:55.882044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:55.882331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:55.882789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:55.885071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:55.963734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:55.965492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d6d5d-ce44-4012-8a92-06e24efcfab3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:55.986160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 906/1849] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:56.199148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.200716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:56.324705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d063a-e800-4460-9c94-60c7145477af admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.333379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.334050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.334327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.334613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] PATH_INFO: `/v3/projects/9ac9baf812524c6da028985f9ab8fff7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.335815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac9baf812524c6da028985f9ab8fff7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.338192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.357680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:56.401449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac9baf812524c6da028985f9ab8fff7', 'request_id': 'req-32a8055f-1df7-4453-9f51-a47582d584f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:56.412280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac9baf812524c6da028985f9ab8fff7', 'request_id': 'req-32a8055f-1df7-4453-9f51-a47582d584f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:56.420751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac9baf812524c6da028985f9ab8fff7', 'request_id': 'req-32a8055f-1df7-4453-9f51-a47582d584f5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:56.431704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] The token cache is being invalidate because project 9ac9baf812524c6da028985f9ab8fff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:56.431984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:56.432737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32a8055f-1df7-4453-9f51-a47582d584f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:56.436768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 944/1850] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:56 2022] DELETE /identity/v3/projects/9ac9baf812524c6da028985f9ab8fff7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:56.439962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.440252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.440821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.462346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:56.463849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:56.471076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:56.572359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 907/1851] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:56.587863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.588148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.588598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.613897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:56.615288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:56.630410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 945/1852] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:56.635088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.648645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3cbfa-eaac-4bec-9068-1e79f9ec3eeb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.664572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.664572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.664572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.664572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.665164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:56.665164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.665164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.682852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 908/1853] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:16:56.687172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.697960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb22d-9bcb-4f3c-b762-dcdb8a71550d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.703426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.704238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.704626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.704987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.705917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.708197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.761068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 946/1854] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:56 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:56.761315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.774615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f7c9-ce39-4058-960d-d2bec3964b62 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.877073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.877691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.877976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.878247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.878902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.880827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.920553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 909/1855] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:56 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:56.924346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.936540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477fc9-6c3e-48cd-9b7d-70de9f4a765b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.941047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.941654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.941932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.942238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.942836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.944446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.955899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 947/1856] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:56.960267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:56.971209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c6ee-2775-43be-86a6-3db228b787bb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:56.976399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:56.977030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:56.977314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:56.977585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:56.978268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:56.979931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455e7ce-2124-426f-b869-fdd82a140315 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:56.991584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 910/1857] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:16:56.996069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.006195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b4c3-9beb-4044-bd32-3af8f3d9231b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.011238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.012035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.012522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.012901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] PATH_INFO: `/v3/projects/26b30833f3be4059abed3d3bd88c956d/users/e1fc72659c60417c98735d763001da8f/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.028822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b30833f3be4059abed3d3bd88c956d, user_id=e1fc72659c60417c98735d763001da8f, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.031685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.048919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 948/1858] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:56 2022] PUT /identity/v3/projects/26b30833f3be4059abed3d3bd88c956d/users/e1fc72659c60417c98735d763001da8f/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:57.060097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.060476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.060893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.131451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] MFA Rules not processed for user `e1fc72659c60417c98735d763001da8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:57.132953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:57.383843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 911/1859] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:57.384392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.403678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.405291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:57.414430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:57.509428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716c9c-08f9-4a50-bb02-4b97800a1794 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.517017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.517758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.518128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.518659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.520730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:57.530015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f439468-13e1-4cdb-bed8-bee8e5b9503d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.536200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.537340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.537887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.538405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.539758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.542499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.586045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 949/1860] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:57 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:57.589929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.613396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.614782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.631734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 912/1861] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:57.645239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:57.699593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba450e-cc9f-4d74-bdf1-5d223c2bca9b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.704689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.705415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.705701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.706574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.707285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.710049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.744642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 950/1862] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:57 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:16:57.749815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.763238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2dec5-a398-4123-a0b4-9d6c61247041 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.768435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.769117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.769393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.769662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.770314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.774556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.789185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 913/1863] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:16:57.804578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.815571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f945fb-bd1d-4b18-8912-90c3ecadaeb2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.831461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.832776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.833147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.833488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] PATH_INFO: `/v3/projects/5941980e4feb441b8093f23c97c650f3/users/8cb3206dce4f4a35beef9a8646a3db61/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.858336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5941980e4feb441b8093f23c97c650f3, user_id=8cb3206dce4f4a35beef9a8646a3db61, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:57.862144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:57.880133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 951/1864] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:57 2022] PUT /identity/v3/projects/5941980e4feb441b8093f23c97c650f3/users/8cb3206dce4f4a35beef9a8646a3db61/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:57.883939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.894838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66571-e713-421d-924c-26b65ee900ba admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:57.937031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:57.940001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:57.997046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:57.997706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:57.997986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:57.998256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:57.998909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.000541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.012712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 914/1865] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:16:58.015964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.026154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c15-2506-4a95-af33-95234ffd8e15 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.028117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb61c83-dd3b-4c85-bfe6-0ed932c99fd1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.030937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.031590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.031869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.032139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] PATH_INFO: `/v3/projects/5941980e4feb441b8093f23c97c650f3/users/8cb3206dce4f4a35beef9a8646a3db61/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.033159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.033799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.034079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.034179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5941980e4feb441b8093f23c97c650f3, user_id=8cb3206dce4f4a35beef9a8646a3db61, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.034352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] PATH_INFO: `/v3/users/a2a1c5ec891a41ed9050e87cb58021ca` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.035499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a1c5ec891a41ed9050e87cb58021ca)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.036782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d8476-9226-4230-aeec-7b7153834c5e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.037518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.062374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 915/1866] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:16:58 2022] PUT /identity/v3/projects/5941980e4feb441b8093f23c97c650f3/users/8cb3206dce4f4a35beef9a8646a3db61/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:58.073597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.073893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.074170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.104381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] MFA Rules not processed for user `8cb3206dce4f4a35beef9a8646a3db61`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:58.105943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.106986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a1c5ec891a41ed9050e87cb58021ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.114187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a1c5ec891a41ed9050e87cb58021ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.119667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:58.128786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a1c5ec891a41ed9050e87cb58021ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.137367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a1c5ec891a41ed9050e87cb58021ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.137999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7fe06e3c-636e-4364-801e-8b726beae1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a1c5ec891a41ed9050e87cb58021ca'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.144438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 952/1867] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:57 2022] DELETE /identity/v3/users/a2a1c5ec891a41ed9050e87cb58021ca => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:58.148761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.149099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.149483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.168892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:58.170096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.174182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:58.179851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 916/1868] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:58.192613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.194078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.232988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 953/1869] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:16:58.243217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.244887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.257350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd052c-5d70-462e-b56b-decd6233e326 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.302372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15967c30-ae75-4cb6-ace0-638ccfc07e46 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.306357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.306969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.307251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.307554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.307657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.308253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.308575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.308843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.316566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.318021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.328348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 954/1870] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:58.370101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.371554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.384244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 917/1871] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:16:58.621860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.626724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.648934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64d612-848e-4d54-8ca3-6b86cbc1c41c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.655728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.656385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.656718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.657004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] PATH_INFO: `/v3/projects/61b8e6df3a7b460280566118e1b55701` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.658389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b8e6df3a7b460280566118e1b55701)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.660359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.731806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b8e6df3a7b460280566118e1b55701', 'request_id': 'req-742c6b80-ee36-4aeb-8972-78c41a6acf6b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.742283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b8e6df3a7b460280566118e1b55701', 'request_id': 'req-742c6b80-ee36-4aeb-8972-78c41a6acf6b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.749352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b8e6df3a7b460280566118e1b55701', 'request_id': 'req-742c6b80-ee36-4aeb-8972-78c41a6acf6b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.764238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] The token cache is being invalidate because project 61b8e6df3a7b460280566118e1b55701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:58.764564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.765371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:58.767389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 955/1872] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:58 2022] DELETE /identity/v3/projects/61b8e6df3a7b460280566118e1b55701 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:58.771447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.784188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898dd60-8f57-4163-a696-fc9865a0ad81 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.820141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:58.860472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:58.861996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:58.927615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:58.930376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.931072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.931349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.931612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] PATH_INFO: `/v3/users/814cb08c186245f685831c490639a1dd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.932871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=814cb08c186245f685831c490639a1dd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.934897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.978292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c6b80-ee36-4aeb-8972-78c41a6acf6b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:58.984072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb3206dce4f4a35beef9a8646a3db61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5941980e4feb441b8093f23c97c650f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:58.985270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:58.985662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:58.986078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:58.986895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:58.987284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:58.988971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:58.994943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814cb08c186245f685831c490639a1dd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.001264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 956/1873] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:16:59.003083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814cb08c186245f685831c490639a1dd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.003847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814cb08c186245f685831c490639a1dd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.005186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.014374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814cb08c186245f685831c490639a1dd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.015080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814cb08c186245f685831c490639a1dd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.016837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4ba3-d321-477a-aa97-be9f67655886 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.021595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb3206dce4f4a35beef9a8646a3db61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5941980e4feb441b8093f23c97c650f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.022221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.022479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.022749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.023381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.025315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.042973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 918/1874] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:58 2022] DELETE /identity/v3/users/814cb08c186245f685831c490639a1dd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:59.070301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 957/1875] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:59 2022] POST /identity/v3/projects => generated 353 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:59.071496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.073129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:59.160785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861f5d-0c3b-4e08-aa9f-c7df746a1ab6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.222699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:59.245675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb3206dce4f4a35beef9a8646a3db61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5941980e4feb441b8093f23c97c650f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.246314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.246598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.246902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.247534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:59.247844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.249509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.267667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 919/1876] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:59.311203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.323060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010679-49aa-4df5-b647-9de0e76be869 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.328199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb3206dce4f4a35beef9a8646a3db61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5941980e4feb441b8093f23c97c650f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.328968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.329217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.329489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] PATH_INFO: `/v3/projects/af58595fdf144658b3874ec1212e4d4e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.330617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorizing `identity:delete_project(project_id=af58595fdf144658b3874ec1212e4d4e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.332696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.344140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.346211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:59.401175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af58595fdf144658b3874ec1212e4d4e', 'request_id': 'req-33363234-0850-4a40-883c-ccbe3febe38e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.412169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af58595fdf144658b3874ec1212e4d4e', 'request_id': 'req-33363234-0850-4a40-883c-ccbe3febe38e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.421289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af58595fdf144658b3874ec1212e4d4e', 'request_id': 'req-33363234-0850-4a40-883c-ccbe3febe38e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.435659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] The token cache is being invalidate because project af58595fdf144658b3874ec1212e4d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:59.435960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.436715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-33363234-0850-4a40-883c-ccbe3febe38e tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.437268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68b7b-834c-44d8-8caf-3d9bb4913fd1 tempest-VolumeQuotaClassesTest-1615886214 tempest-VolumeQuotaClassesTest-1615886214-project-admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.438574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 958/1877] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:59 2022] DELETE /identity/v3/projects/af58595fdf144658b3874ec1212e4d4e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:16:59.501293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.501961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.502383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.502679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] PATH_INFO: `/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.503815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf0dab7b752409bbeb405236e3c2bd7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.505857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.558678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf0dab7b752409bbeb405236e3c2bd7', 'request_id': 'req-42b0f92d-1d3c-43ab-9452-b6045cd5f900'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.564384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf0dab7b752409bbeb405236e3c2bd7', 'request_id': 'req-42b0f92d-1d3c-43ab-9452-b6045cd5f900'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.570691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf0dab7b752409bbeb405236e3c2bd7', 'request_id': 'req-42b0f92d-1d3c-43ab-9452-b6045cd5f900'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.605369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] The token cache is being invalidate because project eaf0dab7b752409bbeb405236e3c2bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:16:59.605369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.605369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.610305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 920/1878] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:16:59 2022] DELETE /identity/v3/projects/eaf0dab7b752409bbeb405236e3c2bd7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:16:59.611958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad61a-6f59-4d4b-8680-88b0132aa34c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.612288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad61a-6f59-4d4b-8680-88b0132aa34c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.612591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad61a-6f59-4d4b-8680-88b0132aa34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.637262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f85ad61a-6f59-4d4b-8680-88b0132aa34c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:59.639189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f85ad61a-6f59-4d4b-8680-88b0132aa34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:59.706775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.708875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:59.722994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 959/1879] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:59.731182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.731746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.732197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.740258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:59.759368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:16:59.762230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:16:59.778094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 960/1880] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:16:59.779422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.791144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92655148-ae41-45f4-b338-65ad920385a3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.797779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.798574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.799424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.799695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.800369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:16:59.801145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.803073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.804154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f92d-1d3c-43ab-9452-b6045cd5f900 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.817224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.817922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 961/1881] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:16:59.821474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.821763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.822047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] PATH_INFO: `/v3/users/e1fc72659c60417c98735d763001da8f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.822544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.823518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fc72659c60417c98735d763001da8f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.825839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.835149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21055b09-9126-4e94-9597-bda4c8f18c69 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:16:59.841679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:16:59.842868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:16:59.843389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:16:59.843947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:16:59.845094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:16:59.847610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:16:59.860212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:16:59.891687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 962/1882] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:16:59 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:16:59.896479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:16:59.900363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fc72659c60417c98735d763001da8f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.912312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fc72659c60417c98735d763001da8f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.914368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fc72659c60417c98735d763001da8f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.944235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fc72659c60417c98735d763001da8f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:16:59.944888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-804fa635-4da5-4287-9535-071438c34b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fc72659c60417c98735d763001da8f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:00.006257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 921/1883] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:16:59 2022] DELETE /identity/v3/users/e1fc72659c60417c98735d763001da8f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:00.007948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0654e-3107-40d9-84b8-dff5aefc0d9f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.011662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.012228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.012809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.056994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:00.058266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:00.099644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.100434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.100973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.101307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.102033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.104303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.137927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 922/1884] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:00.143046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 963/1885] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:16:59 2022] POST /identity/v3/users => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:00.148241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.150095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:00.155916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.158261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:00.166316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7c694-6fd1-48ec-9e42-2145f5563aff None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.173998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.174801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.175166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.175529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.176331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.178626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.193632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 923/1886] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:00.195980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.210828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43685d-1692-4e32-9eb8-58f0e0e236bf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.217235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.217235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.217746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.218101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.218825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.220622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.236440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 924/1887] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:00.237999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.251527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43310-b066-4284-9613-e7a35c7ee114 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.257805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9c89f-4e35-4111-a2a6-14ae05d7c372 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.261923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.262669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.263008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.263771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] PATH_INFO: `/v3/projects/a5b7aa10514c4ef9812c2c851d57ac53/users/83039614370c49bca5cb266bc904b145/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.266879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.267720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.269208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.269764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.276836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:00.281813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b7aa10514c4ef9812c2c851d57ac53, user_id=83039614370c49bca5cb266bc904b145, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.282946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.284730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01226c-96c5-4464-9060-b1798b8ecf5a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.285085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.300406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 964/1888] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:00.308601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 925/1889] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:00 2022] PUT /identity/v3/projects/a5b7aa10514c4ef9812c2c851d57ac53/users/83039614370c49bca5cb266bc904b145/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:00.318737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.319239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.319777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.355489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] MFA Rules not processed for user `83039614370c49bca5cb266bc904b145`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:00.357864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:00.381129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:00.447558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 965/1890] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:00.449670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.531242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f41e-4ff0-4570-a78a-dfe244e321f1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.536985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.537752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.538126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.538490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.539572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.541674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.592569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 926/1891] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:00 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:00.598061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.612340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194e7a7-1e58-4adb-af93-de10f03662d0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.683298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.684003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.684353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.684720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.685512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.687677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.716339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 966/1892] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:00 2022] POST /identity/v3/users => generated 311 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:00.720833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.731819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db4e8-a121-4c2c-82a1-b19531e5530a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.737007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.737790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.738066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.738349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.739011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.740766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.756763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 927/1893] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:00.757981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.769273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6755-b0d3-4fcb-82aa-4ccddd25528c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.773982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.774654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.774931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.775203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] PATH_INFO: `/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1/users/2897b0403c984befa29a8ca935bc20c2/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.807022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d7cb0fffe94009ba2b3e1dc9b418d1, user_id=2897b0403c984befa29a8ca935bc20c2, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.809827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.827235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 967/1894] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:00 2022] PUT /identity/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1/users/2897b0403c984befa29a8ca935bc20c2/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:00.831167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.843635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b41a56-50e3-460c-9053-db837ab6342c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.911895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.912571 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.912862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.913465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.914636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.916638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c4233-e92e-4a21-a90c-fd498922fbf9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.932704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:00.932923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 928/1895] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:00.943906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333a6a-11c8-44cc-9c25-f893c12a88a9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:00.949415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:00.950118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.950406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.950679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] PATH_INFO: `/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1/users/2897b0403c984befa29a8ca935bc20c2/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:00.952632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d7cb0fffe94009ba2b3e1dc9b418d1, user_id=2897b0403c984befa29a8ca935bc20c2, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:00.955534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:00.968587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 968/1896] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:00 2022] PUT /identity/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1/users/2897b0403c984befa29a8ca935bc20c2/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:00.980577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:00.981280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:00.981743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:01.012891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] MFA Rules not processed for user `2897b0403c984befa29a8ca935bc20c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:01.014588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:01.079031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:01.110508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 929/1897] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:01.163338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:01.165685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:01.245156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bea1a-5ff7-4924-9201-d0fcf0bf3210 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:01.253335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:01.253335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:01.253335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:01.253822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:01.260603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:01.262130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:01.267524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:01.269735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:01.292347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 969/1898] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:01.365619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fc46-ad26-4788-a269-c1250c1bf878 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:01.376297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:01.377061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:01.377410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:01.378036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] PATH_INFO: `/v3/projects/26b30833f3be4059abed3d3bd88c956d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:01.379811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b30833f3be4059abed3d3bd88c956d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:01.382200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:01.474135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b30833f3be4059abed3d3bd88c956d', 'request_id': 'req-0f7ee14e-7c89-4667-86ad-6d7191e297ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.482233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b30833f3be4059abed3d3bd88c956d', 'request_id': 'req-0f7ee14e-7c89-4667-86ad-6d7191e297ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.489807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b30833f3be4059abed3d3bd88c956d', 'request_id': 'req-0f7ee14e-7c89-4667-86ad-6d7191e297ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.510972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] The token cache is being invalidate because project 26b30833f3be4059abed3d3bd88c956d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:01.511288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.512234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.514501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 930/1899] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:01 2022] DELETE /identity/v3/projects/26b30833f3be4059abed3d3bd88c956d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:01.518673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:01.539735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9edb6b-ac6b-43eb-ac75-b098a1dd6389 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:01.550504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:01.599879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:01.602097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:01.618291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:01.618949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:01.619230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:01.619504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] PATH_INFO: `/v3/users/8cb3206dce4f4a35beef9a8646a3db61` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:01.621376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb3206dce4f4a35beef9a8646a3db61)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:01.623436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:01.707057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb3206dce4f4a35beef9a8646a3db61'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.725285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee14e-7c89-4667-86ad-6d7191e297ef admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:01.726976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb3206dce4f4a35beef9a8646a3db61'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.737108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:01.737501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:01.737777 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:01.738048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:01.742247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:01.752773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb3206dce4f4a35beef9a8646a3db61'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.774046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb3206dce4f4a35beef9a8646a3db61'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.774805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb3206dce4f4a35beef9a8646a3db61'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:01.783441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 970/1900] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:01 2022] DELETE /identity/v3/users/8cb3206dce4f4a35beef9a8646a3db61 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:01.824957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:01.865469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:01.867889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcaf00-c414-4a4d-91fe-27a2efe22e6b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:01.898063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 931/1901] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:02.184785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.186892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:02.352798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6b40-2956-42a8-b02b-c349df31110e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.358628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.359426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.359779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.360140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] PATH_INFO: `/v3/projects/5941980e4feb441b8093f23c97c650f3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.362223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5941980e4feb441b8093f23c97c650f3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.365449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.415317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:02.433711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5941980e4feb441b8093f23c97c650f3', 'request_id': 'req-fa027beb-e0fe-436e-b1c8-a111555a5643'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:02.456764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5941980e4feb441b8093f23c97c650f3', 'request_id': 'req-fa027beb-e0fe-436e-b1c8-a111555a5643'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:02.464654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5941980e4feb441b8093f23c97c650f3', 'request_id': 'req-fa027beb-e0fe-436e-b1c8-a111555a5643'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:02.478153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] The token cache is being invalidate because project 5941980e4feb441b8093f23c97c650f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:02.478488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:02.479104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fa027beb-e0fe-436e-b1c8-a111555a5643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:02.480933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 971/1902] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:02 2022] DELETE /identity/v3/projects/5941980e4feb441b8093f23c97c650f3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:02.485719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.486003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.486276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.505568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:02.506786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:02.570771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 932/1903] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:02.579220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.579509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.579798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.605805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:02.607465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:02.620212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 972/1904] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:02.621823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.635459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018f168-713f-4408-88a9-4ead0872c1a9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.643154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.643861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.644205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.644603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.645368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:02.645743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.647461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.660187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 933/1905] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:02.665135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.675905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22859ba-b46c-4fad-bcd5-77cee6ff5250 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.680664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.681293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.681572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.681840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.682714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.684725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.715944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 973/1906] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:02 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:02.721561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.733583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa3e1-3b6e-4614-9a11-07c4ec4bbdb2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.799467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.800097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.800374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.800855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.801510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.803495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.828188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 934/1907] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:02 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:02.832423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.843809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850516fc-4bb7-4543-8194-c57fdbad72d0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.848630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.849265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.849681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.850155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.851270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.855034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.882699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 974/1908] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:02.884280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.899504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d210-3f1f-4335-b5c9-37b955f6050a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.904273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.904907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.905188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.905458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.906109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.907691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1293e9-ed4d-44dd-98a8-78c89c5e7a01 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.919787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 935/1909] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:02.923835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:02.934057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2aed-7df2-4650-b3f4-ed704a2a62d4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:02.938735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:02.939727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.939727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.939952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] PATH_INFO: `/v3/projects/abb7d48328834b1c9bc4ec33ca8724bb/users/3124bed21da04ec9944eb66359167122/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:02.958280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb7d48328834b1c9bc4ec33ca8724bb, user_id=3124bed21da04ec9944eb66359167122, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:02.960987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:02.980267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 975/1910] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:02 2022] PUT /identity/v3/projects/abb7d48328834b1c9bc4ec33ca8724bb/users/3124bed21da04ec9944eb66359167122/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:02.991511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:02.991799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:02.992085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.021151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] MFA Rules not processed for user `3124bed21da04ec9944eb66359167122`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:03.022380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:03.052520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:03.096721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 936/1911] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:03.101591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.172064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7045-d0bc-403f-a68c-0dc533bf5a89 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.176833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.177438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.177733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.177997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.178668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.180700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.209456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 976/1912] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:03 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:03.213758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.225940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088645ef-d6c8-4e6f-85f2-164f14e54b0a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.291592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.292496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.294135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.294798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.295088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.297193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.323884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 937/1913] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:03 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:03.328396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.341899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db45b6-8d7b-417a-9ba7-09e5b0d4314c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.347301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.347902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.348149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.348418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.349088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.350846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.379671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 977/1914] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:03.383955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.393975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305481a-f999-469c-a08f-042c3eb3d154 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.400624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.401226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.401650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.401928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] PATH_INFO: `/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf/users/3169c723a22a44c0a714aa9ace6a9999/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.418566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4bb8bdd7524de3ae926bfccfdba8bf, user_id=3169c723a22a44c0a714aa9ace6a9999, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.421425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.436733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 938/1915] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:03 2022] PUT /identity/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf/users/3169c723a22a44c0a714aa9ace6a9999/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:03.440742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.450930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23131-e672-49dc-8bca-937e1939f63d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.483869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:03.513996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.514619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.514895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.515282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.515797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.517436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99a121-aa04-4e42-8641-17efba84d62a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.531303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 978/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:03.533476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.544636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446ee2d-d9f3-4d7f-8a2e-0bf9b6cf0b41 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.549472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.550579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.550963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.551310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] PATH_INFO: `/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf/users/3169c723a22a44c0a714aa9ace6a9999/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.553783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4bb8bdd7524de3ae926bfccfdba8bf, user_id=3169c723a22a44c0a714aa9ace6a9999, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.556865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.569639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 939/1917] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:03 2022] PUT /identity/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf/users/3169c723a22a44c0a714aa9ace6a9999/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:03.597240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.597240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.597240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.627327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] MFA Rules not processed for user `3169c723a22a44c0a714aa9ace6a9999`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:03.628844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:03.701538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 979/1918] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:03.713227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.714954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:03.772797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa2f0-0d0a-4855-a839-01a629aae437 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.776606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.777439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.777752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.778021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.787849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.789668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.799186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 940/1919] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:03.893205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:03.903223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58a4c3-f533-42c2-ae48-456bde4515f6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:03.907618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:03.908734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:03.909001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:03.909289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:03.967425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:03.968915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6b83c-6796-491f-adff-431b03ff2fef service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:03.977785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 980/1920] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:10.392417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:10.393986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:10.416534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:10.468234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161885f9-f591-48a9-8166-d99ed0027303 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:10.473078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:10.473708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:10.473988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:10.474259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] PATH_INFO: `/v3/users/6c5e927518d44e84b6cd187fdfeaa558` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:10.475474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5e927518d44e84b6cd187fdfeaa558)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:10.477494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:10.527961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5e927518d44e84b6cd187fdfeaa558'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:10.535858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5e927518d44e84b6cd187fdfeaa558'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:10.536677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5e927518d44e84b6cd187fdfeaa558'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:10.546102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5e927518d44e84b6cd187fdfeaa558'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:10.546763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5e927518d44e84b6cd187fdfeaa558'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:10.564093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 941/1921] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:10 2022] DELETE /identity/v3/users/6c5e927518d44e84b6cd187fdfeaa558 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:10.568779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:10.569278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:10.569385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:10.590008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:10.591441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:10.652885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:10.666747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 981/1922] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:10.676923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:10.678478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:10.740047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e870-7ca3-4f78-93f4-3d6525bef32c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:10.744045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:10.744703 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:10.744989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:10.745320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:10.754814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:10.756336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58614fa-bac5-45e7-936b-1ba4bb3f6736 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:10.766244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 942/1923] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:10.943759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:10.945681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:10.958945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b408bbb-3aea-4e8d-b08c-c592ec38e688 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:10.963600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:10.964241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:10.964527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:10.964873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] PATH_INFO: `/v3/users/3124bed21da04ec9944eb66359167122` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:10.966044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3124bed21da04ec9944eb66359167122)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:10.968050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:11.019556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3124bed21da04ec9944eb66359167122'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.027444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3124bed21da04ec9944eb66359167122'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.041772 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3124bed21da04ec9944eb66359167122'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.051710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3124bed21da04ec9944eb66359167122'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.053148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3124bed21da04ec9944eb66359167122'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.060306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 982/1924] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:10 2022] DELETE /identity/v3/users/3124bed21da04ec9944eb66359167122 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:11.064791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:11.065128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:11.065412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:11.085753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:11.086984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:11.137143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:11.149939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:11.152051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:11.176672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 943/1925] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:11.186563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:11.188110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:11.244010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136086a5-db42-4db5-9e3a-6aac5180e0cc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:11.251714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:11.252559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:11.252851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:11.253110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] PATH_INFO: `/v3/projects/f03e3971600e4c83913b69a86392402d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:11.254336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9556d4-84e0-4c73-8c5e-0295e25ccfa2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:11.254458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03e3971600e4c83913b69a86392402d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:11.256791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:11.258513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:11.259214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:11.259585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:11.259910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:11.269611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:11.271139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:11.281327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 944/1926] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:11.312711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03e3971600e4c83913b69a86392402d', 'request_id': 'req-c8de3619-51ed-46fa-a9ea-1bb057ba106b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.324219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03e3971600e4c83913b69a86392402d', 'request_id': 'req-c8de3619-51ed-46fa-a9ea-1bb057ba106b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.333304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03e3971600e4c83913b69a86392402d', 'request_id': 'req-c8de3619-51ed-46fa-a9ea-1bb057ba106b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.339812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:11.353192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] The token cache is being invalidate because project f03e3971600e4c83913b69a86392402d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:11.353714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.361186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.362873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 983/1927] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:11 2022] DELETE /identity/v3/projects/f03e3971600e4c83913b69a86392402d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:11.373079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:11.384879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0392-cb85-48c3-8e7c-4f090c37d6f3 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:11.449096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:11.450062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:11.450512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:11.450886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] PATH_INFO: `/v3/users/ea8aac4d2951482fafdde4dbf9100ae0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:11.452324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8aac4d2951482fafdde4dbf9100ae0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:11.454358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:11.525871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8aac4d2951482fafdde4dbf9100ae0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.534614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8aac4d2951482fafdde4dbf9100ae0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.535495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8aac4d2951482fafdde4dbf9100ae0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.551444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8aac4d2951482fafdde4dbf9100ae0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.552915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8aac4d2951482fafdde4dbf9100ae0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:11.579838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 945/1928] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:11 2022] DELETE /identity/v3/users/ea8aac4d2951482fafdde4dbf9100ae0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:11.884710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:11.886725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:11.901453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de3619-51ed-46fa-a9ea-1bb057ba106b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:11.971175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:11.971999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:11.972280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:11.972578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] PATH_INFO: `/v3/projects/abb7d48328834b1c9bc4ec33ca8724bb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:11.973711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb7d48328834b1c9bc4ec33ca8724bb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:11.975637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.017950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.020620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:12.029367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb7d48328834b1c9bc4ec33ca8724bb', 'request_id': 'req-16a21613-7a44-416a-b603-aff22c442796'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.040613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb7d48328834b1c9bc4ec33ca8724bb', 'request_id': 'req-16a21613-7a44-416a-b603-aff22c442796'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.045846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:12.048728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb7d48328834b1c9bc4ec33ca8724bb', 'request_id': 'req-16a21613-7a44-416a-b603-aff22c442796'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.051082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71928a4-af67-4ee6-a4de-aad354b8dd25 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:12.063267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] The token cache is being invalidate because project abb7d48328834b1c9bc4ec33ca8724bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:12.063610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.064701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.067177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 984/1929] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:11 2022] DELETE /identity/v3/projects/abb7d48328834b1c9bc4ec33ca8724bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:12.070598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.072709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:12.133984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:12.134759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:12.135041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:12.135312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] PATH_INFO: `/v3/projects/9591ce42c1e84404a83655d8eb730c60` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:12.136985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9591ce42c1e84404a83655d8eb730c60)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:12.140471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.168750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21613-7a44-416a-b603-aff22c442796 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:12.178789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:12.179557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:12.179979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:12.180255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] PATH_INFO: `/v3/users/3169c723a22a44c0a714aa9ace6a9999` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:12.181434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3169c723a22a44c0a714aa9ace6a9999)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:12.183636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.199723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9591ce42c1e84404a83655d8eb730c60', 'request_id': 'req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.205440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9591ce42c1e84404a83655d8eb730c60', 'request_id': 'req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.213563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9591ce42c1e84404a83655d8eb730c60', 'request_id': 'req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.216890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:12.228288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] The token cache is being invalidate because project 9591ce42c1e84404a83655d8eb730c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:12.228606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.229306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.230949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 946/1930] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/projects/9591ce42c1e84404a83655d8eb730c60 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:12.234826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.236157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:12.239913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3169c723a22a44c0a714aa9ace6a9999'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.249508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3169c723a22a44c0a714aa9ace6a9999'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.262647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3169c723a22a44c0a714aa9ace6a9999'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.272793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3169c723a22a44c0a714aa9ace6a9999'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.273457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3169c723a22a44c0a714aa9ace6a9999'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.281203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 985/1931] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/users/3169c723a22a44c0a714aa9ace6a9999 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:12.307244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846ebd3-9b5f-4aa6-a4fb-2180eca8a5c0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:12.363446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:12.385584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:12.386545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:12.386886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:12.387153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] PATH_INFO: `/v3/users/f11ff4b1c6e5477bb30800ec86bcf43a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:12.389236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11ff4b1c6e5477bb30800ec86bcf43a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:12.391347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.445853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11ff4b1c6e5477bb30800ec86bcf43a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.455751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11ff4b1c6e5477bb30800ec86bcf43a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.456616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11ff4b1c6e5477bb30800ec86bcf43a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.469277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11ff4b1c6e5477bb30800ec86bcf43a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.469878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11ff4b1c6e5477bb30800ec86bcf43a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.488363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 947/1932] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/users/f11ff4b1c6e5477bb30800ec86bcf43a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:12.586550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.589226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:12.630378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501affc4-3a99-449d-b538-4e050196135a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:12.763959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:12.764909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:12.765008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:12.765342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] PATH_INFO: `/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:12.768883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4bb8bdd7524de3ae926bfccfdba8bf)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:12.771411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.853829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.856213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:12.881140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9671eb-228c-4a93-a9d8-7e083084dcd0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:12.924133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4bb8bdd7524de3ae926bfccfdba8bf', 'request_id': 'req-d6236089-f1dc-4272-b3a5-dc96dab12993'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.946629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4bb8bdd7524de3ae926bfccfdba8bf', 'request_id': 'req-d6236089-f1dc-4272-b3a5-dc96dab12993'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.956277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4bb8bdd7524de3ae926bfccfdba8bf', 'request_id': 'req-d6236089-f1dc-4272-b3a5-dc96dab12993'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.960326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:12.963021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:12.963485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:12.963863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] PATH_INFO: `/v3/users/83039614370c49bca5cb266bc904b145` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:12.965235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83039614370c49bca5cb266bc904b145)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:12.967499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:12.973309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] The token cache is being invalidate because project fa4bb8bdd7524de3ae926bfccfdba8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:12.973723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.974564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:12.977181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 986/1933] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/projects/fa4bb8bdd7524de3ae926bfccfdba8bf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:12.977888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:12.979352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.000095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236089-f1dc-4272-b3a5-dc96dab12993 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.019640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.019640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.019640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.019640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] PATH_INFO: `/v3/projects/05143cab443f4cfcb297c4d77e1629be` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.020153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05143cab443f4cfcb297c4d77e1629be)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.022230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.035906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83039614370c49bca5cb266bc904b145'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.045058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83039614370c49bca5cb266bc904b145'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.045980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83039614370c49bca5cb266bc904b145'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.058122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83039614370c49bca5cb266bc904b145'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.058814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f232c7f1-df1a-4dd2-ac92-dccfbcfab344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83039614370c49bca5cb266bc904b145'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.060374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:13.073455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:13.091761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05143cab443f4cfcb297c4d77e1629be', 'request_id': 'req-a756edcf-c0a0-4671-8634-0419e8c89178'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.093247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 948/1934] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/users/83039614370c49bca5cb266bc904b145 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:13.094697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c52b7-6701-479e-aa4e-972bcc1aea96 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.094986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c52b7-6701-479e-aa4e-972bcc1aea96 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.095354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c52b7-6701-479e-aa4e-972bcc1aea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.107474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05143cab443f4cfcb297c4d77e1629be', 'request_id': 'req-a756edcf-c0a0-4671-8634-0419e8c89178'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.115661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05143cab443f4cfcb297c4d77e1629be', 'request_id': 'req-a756edcf-c0a0-4671-8634-0419e8c89178'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.116579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d47c52b7-6701-479e-aa4e-972bcc1aea96 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.117948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d47c52b7-6701-479e-aa4e-972bcc1aea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.130702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] The token cache is being invalidate because project 05143cab443f4cfcb297c4d77e1629be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:13.130990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.131658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.133564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 987/1935] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:12 2022] DELETE /identity/v3/projects/05143cab443f4cfcb297c4d77e1629be => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:13.134282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.135487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.198541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 949/1936] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.200351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee375-5928-42cb-b2ba-06c1a9c9d3a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.200740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee375-5928-42cb-b2ba-06c1a9c9d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.201315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee375-5928-42cb-b2ba-06c1a9c9d3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.226819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1cfee375-5928-42cb-b2ba-06c1a9c9d3a8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.228118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1cfee375-5928-42cb-b2ba-06c1a9c9d3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.231748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756edcf-c0a0-4671-8634-0419e8c89178 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.237183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.237970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.238248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.238566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] PATH_INFO: `/v3/users/03300cd9e4fa418583263336f15b8148` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.242633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 950/1937] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.244400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468d4a1-4303-4f80-ab2c-09931173ba80 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.244723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468d4a1-4303-4f80-ab2c-09931173ba80 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.245011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468d4a1-4303-4f80-ab2c-09931173ba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.254061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03300cd9e4fa418583263336f15b8148)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.257345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.272610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f468d4a1-4303-4f80-ab2c-09931173ba80 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.273958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f468d4a1-4303-4f80-ab2c-09931173ba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.284563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 951/1938] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.284563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.284563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.285206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.324777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03300cd9e4fa418583263336f15b8148'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.325220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.326659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.335268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03300cd9e4fa418583263336f15b8148'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.349211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 952/1939] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.349211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.349211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.367425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03300cd9e4fa418583263336f15b8148'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.379488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03300cd9e4fa418583263336f15b8148'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.381039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-79cc33b4-6388-4161-b531-fb1040f327c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03300cd9e4fa418583263336f15b8148'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:13.388848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 988/1940] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:13 2022] DELETE /identity/v3/users/03300cd9e4fa418583263336f15b8148 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:13.390117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.390400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.391076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.414466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.415700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.420139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3fe32-26b0-4419-b361-e7f06294f583 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.460241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:13.478053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.478685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.478963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.479234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.482385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.492720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 989/1941] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.495317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.497418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.497755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.499011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dc9e0-b911-47d6-95bd-72c601999275 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.509900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 953/1942] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:13.510094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f213a-67ec-4188-9c6b-fceaf49a25ff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.511147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.511430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.511702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.515741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.516605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.517073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.517526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.518346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:13.518815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.520645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.538646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 990/1943] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:13.539160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:13.539160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.540449 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.547541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 954/1944] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:13.549709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.551607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd84dd-7adb-48be-8722-e8e8516e4e74 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.557595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.558289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.558631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.559031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.561239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:13.563543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5c845-86f0-4ab1-a1b8-b18481c9f86a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.564012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.567570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.571111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.572965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.572965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.572965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.577376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.583592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.592481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:13.597882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 991/1945] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:13.602612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.604469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:13.619443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147bc9-0c18-44a5-bd32-25f2e8643f5c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.624133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.624878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.625162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.625468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.629697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 955/1946] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:13.630892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.636885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.640663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.659614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 992/1947] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:13.660899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.674308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32931d8-96f1-4593-92bb-7ff73b4ecce9 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.720309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb28b8-c699-4438-8d47-d0970a946dae admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.742268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.743177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.744463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.744936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.746874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.747264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.747545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.748126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.748473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.749255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.751210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.753288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.790627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 993/1948] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:13.792018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 956/1949] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:13.794271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.795932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.810647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d5b50-d0ef-4e27-9879-3ab4512320e2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.888164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f0aa-2f4b-484b-9d58-e09834c66a6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.897792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.897999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.898791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.899118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.899358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.899464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.899797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.900337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.900889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.902148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.905769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.907794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.913379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 994/1950] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:13.919095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.939141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722ad0b-5e69-43d3-9b74-b9a880138b71 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.942901 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 957/1951] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:13 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:13.943948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.949013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.949934 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.950989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.951933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.953341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.955484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41e4e-db8a-4157-b538-5007ea2da121 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.955718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.961267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:13.962035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:13.962345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:13.962594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:13.963262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:13.965374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:13.976131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 995/1952] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:13.979861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 958/1953] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:13.981171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.985418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:13.994704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb258a8-365b-485b-b76c-e44409900f70 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:13.998805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646236c-eeac-4ae4-8185-b6882e7ec1a7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.008709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.009162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.009434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.009703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] PATH_INFO: `/v3/projects/411ef82f869a4b149cbda770debd158e/users/a1ec3edb25054c45a293b3fc13b6c6de/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.020653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.021325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.021602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.023748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.024467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.026716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.040251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:14.041952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 959/1954] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:14.047545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.051860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=411ef82f869a4b149cbda770debd158e, user_id=a1ec3edb25054c45a293b3fc13b6c6de, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.057936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.061486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c410bc-2540-4f4b-8c37-c70fd76c7faa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.069264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.070340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.070805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.071217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] PATH_INFO: `/v3/projects/2d94d92a8124477d8358077e26c60cfe/users/3a6bae6e54b44a87b01b86e224a8f1e4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.080248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 996/1955] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:13 2022] PUT /identity/v3/projects/411ef82f869a4b149cbda770debd158e/users/a1ec3edb25054c45a293b3fc13b6c6de/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:14.083152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.089817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.091446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d94d92a8124477d8358077e26c60cfe, user_id=3a6bae6e54b44a87b01b86e224a8f1e4, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.094454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49cc03-dfbb-4e80-944f-73153cf4a929 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.117512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 960/1956] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:14 2022] PUT /identity/v3/projects/2d94d92a8124477d8358077e26c60cfe/users/3a6bae6e54b44a87b01b86e224a8f1e4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:14.118788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d4b0-d7d1-4e54-a87c-bc91a4d76188 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.119076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d4b0-d7d1-4e54-a87c-bc91a4d76188 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.119347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d4b0-d7d1-4e54-a87c-bc91a4d76188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.142779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e031d4b0-d7d1-4e54-a87c-bc91a4d76188 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:14.144496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e031d4b0-d7d1-4e54-a87c-bc91a4d76188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.162238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:14.194513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf0811-f01f-42fa-b91b-1893d12bf71a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.232470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 961/1957] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:14.234036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.234476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.234745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.265203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] MFA Rules not processed for user `3a6bae6e54b44a87b01b86e224a8f1e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:14.266431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.278932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.279572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.279851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.280121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] PATH_INFO: `/v3/projects/a5b7aa10514c4ef9812c2c851d57ac53` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.281259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b7aa10514c4ef9812c2c851d57ac53)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.283211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.330993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 962/1958] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:14.331355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b7aa10514c4ef9812c2c851d57ac53', 'request_id': 'req-49cde284-6cf7-42f6-b8f9-df649f9d8e35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.332069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.333444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.341143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b7aa10514c4ef9812c2c851d57ac53', 'request_id': 'req-49cde284-6cf7-42f6-b8f9-df649f9d8e35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.348156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ce5ec-7a3c-49e8-89ed-25dacaec7184 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.348302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b7aa10514c4ef9812c2c851d57ac53', 'request_id': 'req-49cde284-6cf7-42f6-b8f9-df649f9d8e35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.359562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] The token cache is being invalidate because project a5b7aa10514c4ef9812c2c851d57ac53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:14.359857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.360534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.365592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 997/1959] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:14 2022] DELETE /identity/v3/projects/a5b7aa10514c4ef9812c2c851d57ac53 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:14.366569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.368066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.401906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:14.419173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.420154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.420154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.420379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] PATH_INFO: `/v3/projects/61afb8edc9bf4f008c6d7a363350af02` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.421550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61afb8edc9bf4f008c6d7a363350af02)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.423461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.440769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde284-6cf7-42f6-b8f9-df649f9d8e35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.445009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.445680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.446024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.446347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.448185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.460939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.462677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.475462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 998/1960] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:14.476385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.478098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.478332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61afb8edc9bf4f008c6d7a363350af02', 'request_id': 'req-4416475d-a539-4cb3-a177-6263359ff0b6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.481046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:14.492119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61afb8edc9bf4f008c6d7a363350af02', 'request_id': 'req-4416475d-a539-4cb3-a177-6263359ff0b6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.499443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61afb8edc9bf4f008c6d7a363350af02', 'request_id': 'req-4416475d-a539-4cb3-a177-6263359ff0b6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.519111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] The token cache is being invalidate because project 61afb8edc9bf4f008c6d7a363350af02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:14.519568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.521206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.523398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 963/1961] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:14 2022] DELETE /identity/v3/projects/61afb8edc9bf4f008c6d7a363350af02 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:14.524362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.526006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.603520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db211f-bc0f-4049-a662-66c890aa0b97 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.612751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416475d-a539-4cb3-a177-6263359ff0b6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.618126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.618948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.619378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.619665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] PATH_INFO: `/v3/users/2897b0403c984befa29a8ca935bc20c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.620889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2897b0403c984befa29a8ca935bc20c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.622902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.672436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.673363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.673637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.673904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.675895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.684207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2897b0403c984befa29a8ca935bc20c2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.691660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2897b0403c984befa29a8ca935bc20c2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.692427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2897b0403c984befa29a8ca935bc20c2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.703742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2897b0403c984befa29a8ca935bc20c2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.704480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2897b0403c984befa29a8ca935bc20c2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.724632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 964/1962] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:14 2022] DELETE /identity/v3/users/2897b0403c984befa29a8ca935bc20c2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:14.725646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:14.729395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:14.747168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.749153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.758785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 999/1963] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:14.815260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56244b1f-5186-45b8-aa77-c5044080887c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:14.820220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:14.820928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:14.821214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:14.821573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] PATH_INFO: `/v3/users/42c3dcd2154a48e186218750bc335e10` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:14.822767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c3dcd2154a48e186218750bc335e10)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:14.824944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:14.828687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:14.946055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c3dcd2154a48e186218750bc335e10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.961112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c3dcd2154a48e186218750bc335e10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.962394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c3dcd2154a48e186218750bc335e10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.981045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c3dcd2154a48e186218750bc335e10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:14.982045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c3dcd2154a48e186218750bc335e10'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.008141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 965/1964] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:14 2022] DELETE /identity/v3/users/42c3dcd2154a48e186218750bc335e10 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:15.306356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:15.319933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6d9-d3d5-404d-8769-e6dafc08f618 service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:15.358490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:15.390195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:15.392031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.446794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:15.446981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.447208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.447503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] PATH_INFO: `/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.449168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d7cb0fffe94009ba2b3e1dc9b418d1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:15.456179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:15.523205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb7ca5-1504-4fea-b083-a82baf3c4c6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:15.535781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d7cb0fffe94009ba2b3e1dc9b418d1', 'request_id': 'req-80445ec0-c9e6-47f1-9d5c-354c3095e998'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.547337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d7cb0fffe94009ba2b3e1dc9b418d1', 'request_id': 'req-80445ec0-c9e6-47f1-9d5c-354c3095e998'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.556088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d7cb0fffe94009ba2b3e1dc9b418d1', 'request_id': 'req-80445ec0-c9e6-47f1-9d5c-354c3095e998'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.576893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] The token cache is being invalidate because project 46d7cb0fffe94009ba2b3e1dc9b418d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:15.577262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.578680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-80445ec0-c9e6-47f1-9d5c-354c3095e998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.580677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1000/1965] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:15 2022] DELETE /identity/v3/projects/46d7cb0fffe94009ba2b3e1dc9b418d1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:15.581992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ca62-9758-41e1-a380-7960fa60ae64 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.587243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ca62-9758-41e1-a380-7960fa60ae64 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.587576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ca62-9758-41e1-a380-7960fa60ae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.618563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:15.619242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.619560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.619895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] PATH_INFO: `/v3/projects/52a43d9081104fb39f8da33517ee0965` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.621285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a43d9081104fb39f8da33517ee0965)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:15.624092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:15.647725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-88e7ca62-9758-41e1-a380-7960fa60ae64 None None] MFA Rules not processed for user `a1ec3edb25054c45a293b3fc13b6c6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:15.652864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-88e7ca62-9758-41e1-a380-7960fa60ae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.668345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:15.688815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a43d9081104fb39f8da33517ee0965', 'request_id': 'req-d0906fad-2240-4cda-be93-006a7b4d96f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.695858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a43d9081104fb39f8da33517ee0965', 'request_id': 'req-d0906fad-2240-4cda-be93-006a7b4d96f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.702884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a43d9081104fb39f8da33517ee0965', 'request_id': 'req-d0906fad-2240-4cda-be93-006a7b4d96f7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.721989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] The token cache is being invalidate because project 52a43d9081104fb39f8da33517ee0965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:15.722290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.722998 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d0906fad-2240-4cda-be93-006a7b4d96f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:15.728630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 966/1966] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:15 2022] DELETE /identity/v3/projects/52a43d9081104fb39f8da33517ee0965 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:15.730030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.730841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.731134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.753853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:15.755151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.774164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1001/1967] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:15.775429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f47cc-fb12-47fa-ad99-1b231113e1ae None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.775720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f47cc-fb12-47fa-ad99-1b231113e1ae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.776008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f47cc-fb12-47fa-ad99-1b231113e1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.797480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-952f47cc-fb12-47fa-ad99-1b231113e1ae None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:15.798771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-952f47cc-fb12-47fa-ad99-1b231113e1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.822766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 967/1968] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:15.823604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:15.824935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.837343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449fe43-98b9-4a3b-8786-ec615c8e0d31 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:15.840414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:15.842823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:15.843473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.843745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.844011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.844677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:15.846569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01122d96-4eae-4658-801f-bb70c4daea5c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:15.869960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1002/1969] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:15.871239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.871582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.871860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.878292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 968/1970] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:15.878292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.878292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.878605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.894883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:15.896087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.898112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:15.899269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:15.961337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 969/1971] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:15.962592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:15.963195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1003/1972] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:15.967703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:15.972179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e617c7-8947-45df-b3f9-f7c286265a4d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:15.977447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:15.978059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.978250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24041d-a090-43e8-85b5-b2938d753fec None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:15.978333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.978582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.979361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:15.981353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:15.983714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:15.984322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:15.984591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:15.985316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:15.986078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:15.986278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:15.988003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.014984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1004/1973] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:16.015966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.019946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 970/1974] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:15 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:16.021444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.028074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017744c-4cc5-45e8-9f41-2c9b50bbb4d1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.034099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.034876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.035165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.035438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.035745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef84ca1-5319-4e4d-af32-acff6227d188 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.036103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:16.036454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.038285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.041168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.041817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.042125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.042442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.043330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.045802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.061489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1005/1975] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:16.061489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.070244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12c97e-3325-405c-bbce-ddd2dd23109c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.070598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:16.077979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.078712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.079041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.079532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.080266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.082323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.085097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 971/1976] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:16.089336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.093454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1006/1977] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:16.094229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.103865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3312db-24f1-4535-af66-76a4ef42a054 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.175131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd237b3e-436a-4f90-8356-50187140a29c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.180588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.181294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.181576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.181871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.182514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.182615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.183132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.183408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.183673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.184332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.184524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.186278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.212902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1007/1978] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:16.212902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.213539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 972/1979] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:16.214259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.227752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9096d8-1e4d-4bff-851b-1ae34cae45e6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.264179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:16.286555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971d5b-fa41-42e0-b742-e21d552c5b4d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.291551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.292190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.292344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.292722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.293001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.293108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.293352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.293647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] PATH_INFO: `/v3/projects/3efff253557c45d295e2381e0a9b4454/users/1dcc63dd659b44a98ee0e01de7e458c7/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.293755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.295352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.306225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 973/1980] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:16.307010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.310970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efff253557c45d295e2381e0a9b4454, user_id=1dcc63dd659b44a98ee0e01de7e458c7, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.313825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.316550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8dbf4-d7d4-4fe6-808e-b83e6c14e623 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.321933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.322793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.323075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.323350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.324000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.326308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.330063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1008/1981] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:16 2022] PUT /identity/v3/projects/3efff253557c45d295e2381e0a9b4454/users/1dcc63dd659b44a98ee0e01de7e458c7/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:16.333844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.349810 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 974/1982] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:16.351156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.362697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172e6f-28c9-4e12-9106-2d8be419071b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.399061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0de9bc-cf5c-4ca7-8ef1-ab5010b18a7b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.404423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.405074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.405357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.405627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.406278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.408038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.426283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.427088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.427382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.427742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.428554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.430683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.435911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1009/1983] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:16.437205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.437723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:16.445938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 975/1984] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:16.446957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.456483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f689d-9751-4595-b9f8-aa26a330da93 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.458944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e58d7-09f3-4660-9d53-99dcfd226fae admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.461758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.462457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.462748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.463014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.463536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.463664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.464200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.464475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.464775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] PATH_INFO: `/v3/projects/0c15b0fbd9794f07975258a5e17bdc51/users/07bf5c7d6b3e49928582ea824a7d77b3/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.465383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.480602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1010/1985] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:16.480602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.482811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c15b0fbd9794f07975258a5e17bdc51, user_id=07bf5c7d6b3e49928582ea824a7d77b3, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.485561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.488936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fc57-5eec-42d8-929e-b874b7c96cd2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.494962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.495649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.495926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.496228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] PATH_INFO: `/v3/projects/3efff253557c45d295e2381e0a9b4454/users/1dcc63dd659b44a98ee0e01de7e458c7/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.498242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efff253557c45d295e2381e0a9b4454, user_id=1dcc63dd659b44a98ee0e01de7e458c7, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.500906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b2d11-2331-4391-8c52-121e190250f3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.512539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 976/1986] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:16 2022] PUT /identity/v3/projects/0c15b0fbd9794f07975258a5e17bdc51/users/07bf5c7d6b3e49928582ea824a7d77b3/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:16.512539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.520936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1011/1987] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:16 2022] PUT /identity/v3/projects/3efff253557c45d295e2381e0a9b4454/users/1dcc63dd659b44a98ee0e01de7e458c7/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:16.525816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.526105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.526384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.559342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] MFA Rules not processed for user `07bf5c7d6b3e49928582ea824a7d77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:16.561087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:16.594746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df1ca2-1993-4233-8800-f76469f2b165 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.649467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1012/1988] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:16.669220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.671761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:16.681562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.682539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.682671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.682966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.683706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.685545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.696255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 977/1989] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:16.700701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.706454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:16.712167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2e11b-c02b-4ebf-9f32-2c920b01d391 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.718176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.719020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.719855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.719855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] PATH_INFO: `/v3/projects/6a42a329a2124826877337972883d8fe/users/dbfa69c587e547f88f3f56111e8014af/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.738661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a42a329a2124826877337972883d8fe, user_id=dbfa69c587e547f88f3f56111e8014af, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.742945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e82ed-f7e1-47b3-af0e-3754c30b6a04 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.759554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98ba74-225e-423a-8951-545bd8fed6ff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:16.769695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 978/1990] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:16 2022] PUT /identity/v3/projects/6a42a329a2124826877337972883d8fe/users/dbfa69c587e547f88f3f56111e8014af/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:16.784381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.788007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.788337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.829389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] MFA Rules not processed for user `dbfa69c587e547f88f3f56111e8014af`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:16.830610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:16.837112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:16.838438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:16.838438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:16.838629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:16.882723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:16.923648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 979/1991] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:16.925173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:16.939870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:16.977114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:16.989906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1013/1992] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:17.015838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.081685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692b28f-4921-4910-b054-74b33f513bc7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.091331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.092259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.092583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.093156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.094135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.096552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.131725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3a5b5-e0e3-41a6-a976-2e6b67657548 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.133634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 980/1993] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:16 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:17.138598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.170239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:17.224793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16519f4-ff47-4ba2-8adb-3ccbac96149c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.225052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.227382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.227382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.227382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] PATH_INFO: `/v3/users/07bf5c7d6b3e49928582ea824a7d77b3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.228443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bf5c7d6b3e49928582ea824a7d77b3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.230927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.232872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.233589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.233930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.234278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.234935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.237663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.271046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 981/1994] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:17 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:17.272899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.284625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e17b9e-88fd-4164-94f5-9b77f7ca50cd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.291757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.292694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.292959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.293300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.293684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bf5c7d6b3e49928582ea824a7d77b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:17.293957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.295742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.305149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bf5c7d6b3e49928582ea824a7d77b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:17.315776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 982/1995] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:17.319779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.322227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bf5c7d6b3e49928582ea824a7d77b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:17.330824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44c6e5-0e12-404d-b327-914d5c0c53fd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.331012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bf5c7d6b3e49928582ea824a7d77b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:17.331658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-807f0939-ad40-4805-95dc-d670630d2a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bf5c7d6b3e49928582ea824a7d77b3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:17.336740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.337336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.337738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.338017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] PATH_INFO: `/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225/users/55ed6fbc72e54bd8b5b8fbd0539f526d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.338674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1014/1996] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:17 2022] DELETE /identity/v3/users/07bf5c7d6b3e49928582ea824a7d77b3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:17.339776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.340092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.340366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.358048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1a99e29a7b42a9967e65bb27ee1225, user_id=55ed6fbc72e54bd8b5b8fbd0539f526d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.361751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e842f5-14a8-45af-9628-1456d4d0e9d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.377259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] MFA Rules not processed for user `1dcc63dd659b44a98ee0e01de7e458c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:17.381117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 983/1997] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:17 2022] PUT /identity/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225/users/55ed6fbc72e54bd8b5b8fbd0539f526d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:17.381313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.381496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.381496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.381496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.401387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:17.403606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.419336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:17.462854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1015/1998] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:17.464069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.465832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.474475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 984/1999] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:17.480319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.482037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.482618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27b391-474c-41c2-b632-763a4e3e55a5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.489573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.490626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.491024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.491398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.492199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.494328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.512717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1016/2000] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:17.513828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.515551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.550676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:17.561952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034a6c7-160f-48a2-98cd-0cab9df5d02b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.567125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.567909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.568188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.568681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.570617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.592211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceff9b9-fadf-4593-a430-b6c7f6449803 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.595872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.596482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.596784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.597042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.606591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.608024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.617727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1017/2001] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:17.618569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.627912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696a36-b2bf-4b05-b627-b3949f64fa0e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.632923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.633579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.633855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.634458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] PATH_INFO: `/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225/users/55ed6fbc72e54bd8b5b8fbd0539f526d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.634907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.636129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1a99e29a7b42a9967e65bb27ee1225, user_id=55ed6fbc72e54bd8b5b8fbd0539f526d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.637230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.638974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48edd-0f53-4a0e-9818-3003e2fb605f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.650178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1018/2002] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:17 2022] PUT /identity/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225/users/55ed6fbc72e54bd8b5b8fbd0539f526d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:17.652434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 985/2003] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:17.661994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.662279 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.662551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.697784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] MFA Rules not processed for user `55ed6fbc72e54bd8b5b8fbd0539f526d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:17.699026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.791701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1019/2004] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:17.805743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.807289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:17.833008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7e436-e15b-47cd-a84e-8e2b09f7b748 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:17.888340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:17.889270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:17.889951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:17.889951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:17.900225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:17.901739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:17.912033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 986/2005] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:17.970601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:17.973429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.041283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.052842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371443f-f927-4dee-8f82-31d4627f8d8c service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.056965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.059293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.059293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.059293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.059973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.074446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:18.090200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01576ba0-581e-48f5-b6bf-a87abc3c8335 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.096109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.096858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.097203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.097537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] PATH_INFO: `/v3/projects/0c15b0fbd9794f07975258a5e17bdc51` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.099754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c15b0fbd9794f07975258a5e17bdc51)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.101827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.147542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.149105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b6eba-49ec-41c5-be5f-d36fe89b636a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.159641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 987/2006] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:18.163465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:18.170608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c15b0fbd9794f07975258a5e17bdc51', 'request_id': 'req-3770846f-505b-4054-b441-e35b15dcbf5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:18.183497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c15b0fbd9794f07975258a5e17bdc51', 'request_id': 'req-3770846f-505b-4054-b441-e35b15dcbf5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:18.193060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c15b0fbd9794f07975258a5e17bdc51', 'request_id': 'req-3770846f-505b-4054-b441-e35b15dcbf5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:18.205068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] The token cache is being invalidate because project 0c15b0fbd9794f07975258a5e17bdc51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:18.205436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:18.206203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3770846f-505b-4054-b441-e35b15dcbf5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:18.207936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1020/2007] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:17 2022] DELETE /identity/v3/projects/0c15b0fbd9794f07975258a5e17bdc51 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:18.212968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.213303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.213579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.236200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:18.238069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.314815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 988/2008] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:18.322286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.322786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.322891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.342804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:18.343922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.352762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1021/2009] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:18.357330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.367565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf90ce-5430-4c32-8af8-b465918c0a21 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.372879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.373479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.373751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.374018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.374671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:18.374967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.376542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.389016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 989/2010] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:17:18.393623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.403535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f4523-6a92-4abb-be92-5a39f94b9e3e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.409716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.410314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.410584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.410849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.411475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.413406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.466127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1022/2011] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:18 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:18.467394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.477693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f91f4-6e04-46de-b02b-782fe664b655 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.535233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.535856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.536139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.536408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.537079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.539365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.565953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 990/2012] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:18 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:18.570149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.581246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc40555-73b8-4bf8-bae9-e26ef7338c27 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.587501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.588111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.588392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.588782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.589430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.591051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.601917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1023/2013] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:18.606146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.616203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303670-8090-443e-a6f9-3a52793dbf81 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.620849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.621476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.621740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.622087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.622670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.624264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edaf8e-f273-4c97-af83-f659b9661cd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.635091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 991/2014] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:18.639429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.649699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe2ba4-9cb2-4948-b64d-304efb1a4c1b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.654526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.655185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.655469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.655739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] PATH_INFO: `/v3/projects/cd98670ba9314aba9a92fe1ef7b26388/users/acadef019281457dbdb757118a129f2d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.671756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd98670ba9314aba9a92fe1ef7b26388, user_id=acadef019281457dbdb757118a129f2d, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.674495 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.689697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1024/2015] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:18 2022] PUT /identity/v3/projects/cd98670ba9314aba9a92fe1ef7b26388/users/acadef019281457dbdb757118a129f2d/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:18.700159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.700443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.700741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.729876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] MFA Rules not processed for user `acadef019281457dbdb757118a129f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:18.731237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.741930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:18.808638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 992/2016] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:18.819927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:18.821753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:18.880597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5e376-dfa7-4f75-8c04-c083bb597ecc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:18.884459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:18.885096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:18.885373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:18.885637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:18.894091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:18.895534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:18.904308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1025/2017] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:19.072951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:19.075028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:19.139827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a66f-e900-4ce8-9e11-718e2a8d852c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:19.143291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:19.143900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:19.144173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:19.144437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:19.146166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:19.211843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:19.214147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bde4-bf69-4c6a-a833-990c48b36ee9 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:19.232800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 993/2018] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:21.015041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:21.016491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:21.084174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c48702-9af7-4651-88cb-afcd1330f246 service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:21.088788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:21.089420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:21.089696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:21.089972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] PATH_INFO: `/v3/users/dbfa69c587e547f88f3f56111e8014af` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:21.091090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfa69c587e547f88f3f56111e8014af)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:21.093091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:21.107119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:21.145375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfa69c587e547f88f3f56111e8014af'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.152097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfa69c587e547f88f3f56111e8014af'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.163695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfa69c587e547f88f3f56111e8014af'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.175997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfa69c587e547f88f3f56111e8014af'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.176780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfa69c587e547f88f3f56111e8014af'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.183224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1026/2019] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:21 2022] DELETE /identity/v3/users/dbfa69c587e547f88f3f56111e8014af => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:21.188090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:21.188566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:21.188677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:21.208692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:21.209914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:21.271736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 994/2020] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:21.281886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:21.283480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:21.342886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55794dc-a05e-452d-af9a-faa0043bcbb7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:21.346845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:21.347460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:21.347733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:21.348005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:21.357403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:21.358868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:21.369578 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1027/2021] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:21.662981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:21.664866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:21.672198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:21.677633 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d19bae-f40e-4d75-9e91-6ec9d078f9db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:21.692436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:21.693114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:21.693388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:21.693667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] PATH_INFO: `/v3/projects/6a42a329a2124826877337972883d8fe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:21.696429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a42a329a2124826877337972883d8fe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:21.698429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:21.772194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a42a329a2124826877337972883d8fe', 'request_id': 'req-f0b7baee-8311-4898-a56d-a00d186681e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.778808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a42a329a2124826877337972883d8fe', 'request_id': 'req-f0b7baee-8311-4898-a56d-a00d186681e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.785861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a42a329a2124826877337972883d8fe', 'request_id': 'req-f0b7baee-8311-4898-a56d-a00d186681e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.805007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] The token cache is being invalidate because project 6a42a329a2124826877337972883d8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:21.805383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.806085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.815223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 995/2022] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:21 2022] DELETE /identity/v3/projects/6a42a329a2124826877337972883d8fe => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:21.816448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:21.827905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacfeb7-4cea-435b-bfe2-9c10be1ba7ca service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:21.896903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:21.897555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:21.897826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:21.898125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] PATH_INFO: `/v3/users/55ed6fbc72e54bd8b5b8fbd0539f526d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:21.899253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ed6fbc72e54bd8b5b8fbd0539f526d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:21.901221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:21.949522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:21.959227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ed6fbc72e54bd8b5b8fbd0539f526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.966945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ed6fbc72e54bd8b5b8fbd0539f526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.980152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ed6fbc72e54bd8b5b8fbd0539f526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.989024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ed6fbc72e54bd8b5b8fbd0539f526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.989777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-965a20dc-5173-4c6e-bb56-0d327abae113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ed6fbc72e54bd8b5b8fbd0539f526d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:21.996894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1028/2023] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:21 2022] DELETE /identity/v3/users/55ed6fbc72e54bd8b5b8fbd0539f526d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:22.291095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.292730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:22.306741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7baee-8311-4898-a56d-a00d186681e1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.366224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.366870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.367145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.367420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] PATH_INFO: `/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.368599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1a99e29a7b42a9967e65bb27ee1225)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.370602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.426140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1a99e29a7b42a9967e65bb27ee1225', 'request_id': 'req-122f0add-fb39-4351-a037-45b97313ad91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:22.432595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1a99e29a7b42a9967e65bb27ee1225', 'request_id': 'req-122f0add-fb39-4351-a037-45b97313ad91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:22.439472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1a99e29a7b42a9967e65bb27ee1225', 'request_id': 'req-122f0add-fb39-4351-a037-45b97313ad91'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:22.459925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] The token cache is being invalidate because project 9f1a99e29a7b42a9967e65bb27ee1225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:22.460217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:22.460945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-122f0add-fb39-4351-a037-45b97313ad91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:22.462683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 996/2024] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:22 2022] DELETE /identity/v3/projects/9f1a99e29a7b42a9967e65bb27ee1225 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:22.469012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.469497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.469581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.492776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:22.494013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:22.555632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1029/2025] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:22.563892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.564182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.564449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.594044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:22.596016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:22.604786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 997/2026] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:22.609418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.620324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7a8a7-e24c-47e7-a0b6-2667af0a84ad None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.624986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.625585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.625862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.626131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.626783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:22.627084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.628719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.640293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1030/2027] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:22.644967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.654802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bede5-9605-42f4-bb90-724ebd25da1e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.659417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.660018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.660295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.660597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.661251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.663183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.668952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:22.695171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 998/2028] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:22 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:22.699605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.710071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26052c-cd3b-4a17-b9ca-363059ae936e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.775788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.776390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.776699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.776963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.777598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.779540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.803084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1031/2029] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:22 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:22.810198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.821192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9244d2-9f4c-439f-9290-fc7a9c31ca9b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.826180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.826673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.826929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.827224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.827842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.829471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.840171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 999/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:22.844317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.855604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667100ec-ce9e-4aa6-8c17-ba8408dd9238 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.860932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.861540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.861817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.862179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.862735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.864403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9dc4a-8bfc-4e50-b2be-0dfabffae1e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.876137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1032/2031] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:22.880616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:22.890543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333dabb-194e-4bfb-9bf2-71e7663df292 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:22.895265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:22.895924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.896199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.896464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] PATH_INFO: `/v3/projects/b3a1c0e01f8e4136ba2749b704c1887b/users/bd1d74b254564c01b189d0ea94124632/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.911135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a1c0e01f8e4136ba2749b704c1887b, user_id=bd1d74b254564c01b189d0ea94124632, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:22.913799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:22.930234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1000/2032] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:22 2022] PUT /identity/v3/projects/b3a1c0e01f8e4136ba2749b704c1887b/users/bd1d74b254564c01b189d0ea94124632/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:22.940422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:22.940749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:22.941008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:22.990631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] MFA Rules not processed for user `bd1d74b254564c01b189d0ea94124632`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:22.991960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:23.012348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:23.090886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1033/2033] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:23.095522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.214417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3a66-617f-4b5b-9a0a-ae849f5ef241 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.219252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.219938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.220279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.220649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.221370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.223444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.253425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1001/2034] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:23 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:23.257227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.268515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218ebe9-de40-48f2-90c3-5c2a77ca6b13 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.329643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.330283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.330559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.330829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.331440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.333455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.357705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1034/2035] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:23 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:23.362047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.372860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80affd3c-ede6-4238-89df-610737d70268 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.378194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.378813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.379092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.379359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.380019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.381685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.393112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1002/2036] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:23.397684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.408213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201de803-ded4-4ffe-a7be-b6f803d19a9f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.413595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.414586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.414863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.415133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] PATH_INFO: `/v3/projects/22c3645a3531439790dfdfb2148f6bfb/users/190bf1c397de41738a23e169416eb2d5/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.431328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c3645a3531439790dfdfb2148f6bfb, user_id=190bf1c397de41738a23e169416eb2d5, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.434021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.449660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1035/2037] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:23 2022] PUT /identity/v3/projects/22c3645a3531439790dfdfb2148f6bfb/users/190bf1c397de41738a23e169416eb2d5/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:23.453944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.464176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821eb7-8ac0-4bfd-9edd-64cee29ae454 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.532308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.532948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.533199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.533467 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.534113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.535771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a2554-36c7-440d-b42b-47de5f1f0be0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.547056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1003/2038] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:23.551707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.562774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316526c2-5f91-45cd-ac7f-b5a97269f159 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.568799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.569451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.569737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.570009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] PATH_INFO: `/v3/projects/22c3645a3531439790dfdfb2148f6bfb/users/190bf1c397de41738a23e169416eb2d5/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.572061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c3645a3531439790dfdfb2148f6bfb, user_id=190bf1c397de41738a23e169416eb2d5, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.574725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.586203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1036/2039] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:23 2022] PUT /identity/v3/projects/22c3645a3531439790dfdfb2148f6bfb/users/190bf1c397de41738a23e169416eb2d5/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:23.598030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.598319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.598593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.626811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] MFA Rules not processed for user `190bf1c397de41738a23e169416eb2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:23.628191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:23.639994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:23.703440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1004/2040] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:23.715275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.717097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:23.773951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcb88-89a8-4547-9a49-c3b57ccc0d59 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.778026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.778633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.778918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.779183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.788263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.790101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:23.799487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1037/2041] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:23.911005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:23.922801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e3375-ce8a-4b79-b9e8-a15464943582 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:23.927769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:23.928394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:23.928883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:23.929148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:23.991341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:23.992736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce280dd6-b8dc-4d98-882f-553e6a5620db service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:24.001751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1005/2042] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:27.682714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:27.688472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:27.750353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ab35-f7c4-4bd7-83d9-38d30e348cbb service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:27.755260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:27.755894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:27.756167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:27.756433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] PATH_INFO: `/v3/users/bd1d74b254564c01b189d0ea94124632` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:27.757755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1d74b254564c01b189d0ea94124632)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:27.759768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:27.813367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1d74b254564c01b189d0ea94124632'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:27.820970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1d74b254564c01b189d0ea94124632'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:27.834611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1d74b254564c01b189d0ea94124632'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:27.845278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1d74b254564c01b189d0ea94124632'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:27.845986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1d74b254564c01b189d0ea94124632'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:27.852030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1038/2043] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:27 2022] DELETE /identity/v3/users/bd1d74b254564c01b189d0ea94124632 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:27.856914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:27.857338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:27.857629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:27.882189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:27.883578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:27.954475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1006/2044] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:27.965077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:27.966596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:28.040863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39a936-6f46-4290-ad46-a994abc75963 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:28.045754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:28.046182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:28.046272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:28.046547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:28.060635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:28.062495 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:28.085143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1039/2045] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:28.395340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:28.397023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:28.418230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1b271-05b9-4537-b004-f9dd5f3df202 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:28.424456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:28.425354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:28.425750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:28.426084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] PATH_INFO: `/v3/projects/b3a1c0e01f8e4136ba2749b704c1887b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:28.427297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a1c0e01f8e4136ba2749b704c1887b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:28.430221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:28.477351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:28.483883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a1c0e01f8e4136ba2749b704c1887b', 'request_id': 'req-9f24d364-335d-4010-8ed0-f84ad9417eb6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.489387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a1c0e01f8e4136ba2749b704c1887b', 'request_id': 'req-9f24d364-335d-4010-8ed0-f84ad9417eb6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.495994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a1c0e01f8e4136ba2749b704c1887b', 'request_id': 'req-9f24d364-335d-4010-8ed0-f84ad9417eb6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.511176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] The token cache is being invalidate because project b3a1c0e01f8e4136ba2749b704c1887b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:28.511461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.512149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.513784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1007/2046] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:28 2022] DELETE /identity/v3/projects/b3a1c0e01f8e4136ba2749b704c1887b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:28.518148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:28.528146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfe83f-9228-42ec-bfe7-5da554514b37 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:28.557312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:28.599325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:28.599968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:28.600248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:28.600544 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] PATH_INFO: `/v3/users/190bf1c397de41738a23e169416eb2d5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:28.601713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190bf1c397de41738a23e169416eb2d5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:28.603655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:28.654528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190bf1c397de41738a23e169416eb2d5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.662123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190bf1c397de41738a23e169416eb2d5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.675558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190bf1c397de41738a23e169416eb2d5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.684092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190bf1c397de41738a23e169416eb2d5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.684875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bbf8acd-56cb-4177-b5d9-736956b83be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190bf1c397de41738a23e169416eb2d5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:28.690734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1040/2047] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:28 2022] DELETE /identity/v3/users/190bf1c397de41738a23e169416eb2d5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:29.179469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.180930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:29.194683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d364-335d-4010-8ed0-f84ad9417eb6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.259469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.260203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.260584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.261053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] PATH_INFO: `/v3/projects/22c3645a3531439790dfdfb2148f6bfb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.262283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c3645a3531439790dfdfb2148f6bfb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.264694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.318347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c3645a3531439790dfdfb2148f6bfb', 'request_id': 'req-888a2668-f12c-410e-af4f-8490ed71d00d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:29.324061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c3645a3531439790dfdfb2148f6bfb', 'request_id': 'req-888a2668-f12c-410e-af4f-8490ed71d00d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:29.330765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c3645a3531439790dfdfb2148f6bfb', 'request_id': 'req-888a2668-f12c-410e-af4f-8490ed71d00d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:29.347536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] The token cache is being invalidate because project 22c3645a3531439790dfdfb2148f6bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:29.347826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:29.348457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-888a2668-f12c-410e-af4f-8490ed71d00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:29.350013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1008/2048] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:29 2022] DELETE /identity/v3/projects/22c3645a3531439790dfdfb2148f6bfb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:29.355383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.355682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.355984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.385146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:29.386540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:29.455121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1041/2049] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:29.462891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.463177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.463450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.483386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:29.484631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:29.498977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1009/2050] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:29.500422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.517539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7da1b-b03c-405e-9c32-3e767459c121 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.522918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.523590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.523973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.524555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.525236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:29.525543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.527195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.544543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1042/2051] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:29.546019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.555620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f4010-5f7f-4f1c-95d9-b4bf2af8d53f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.560308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.561070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.561411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.561746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.562516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.564624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.600559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1010/2052] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:29.602141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.612455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03a6a-3969-43ed-a787-083d0f8bf4b6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.643134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:29.673250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.673871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.674151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.674427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.675063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.677034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.698814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1043/2053] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:29.703068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.713659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a3bab-2afc-4d70-a1ae-aada1cfaaff4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.721657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.722305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.722591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.722862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.723533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.725295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.736540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1011/2054] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:29.740538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.751280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74da00-e8c6-40dc-9f17-4c6d79ef0e3d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.757102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.757720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.758001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.758273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.758925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.760701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f7db5-2cce-4501-aace-529300091635 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.771388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1044/2055] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:29.775744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:29.786629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6da7cc-0ba3-4d2c-b60e-b405f8007373 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:29.793248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:29.793973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.794313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.794587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] PATH_INFO: `/v3/projects/8c187a7feef44c37a88027de76d979fd/users/64207a6fa10843eba27e4e0b7ad21880/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.810699 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c187a7feef44c37a88027de76d979fd, user_id=64207a6fa10843eba27e4e0b7ad21880, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:29.813476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:29.816471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:29.829916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1012/2056] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:29 2022] PUT /identity/v3/projects/8c187a7feef44c37a88027de76d979fd/users/64207a6fa10843eba27e4e0b7ad21880/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:29.841222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:29.841601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:29.842065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:29.870069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] MFA Rules not processed for user `64207a6fa10843eba27e4e0b7ad21880`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:29.871390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:29.939067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1045/2057] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:29.943787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.013586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be72ab-4a91-4916-8751-3d0066fc4fc2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.019343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.019969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.020250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.020570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.021389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.023393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.061287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1013/2058] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:29 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:30.065758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.077491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753e5e0-4915-400e-a6f2-26c3b0749e05 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.138737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.139443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.139717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.139983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.140655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.142592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.166024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1046/2059] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:30 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:30.170176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.182548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542c467-360b-40d5-9ea8-36f74404b0a7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.187933 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.188644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.188922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.189216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.189847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.191499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.203055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1014/2060] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:30.207014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.218161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fd97f-cdaa-4b64-b3cd-dab8bb08c027 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.223120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.223726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.224003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.224273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.224964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.226594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52151de9-0539-408d-9870-d05d4e0d4615 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.237717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1047/2061] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:30.241652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.252547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321023e3-7f36-4592-a8d4-6019e7947285 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.258123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.258794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.259070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.259340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] PATH_INFO: `/v3/projects/24dd998ca5d24575a2269da173be3a29/users/8f97703389904a5aa2131fbece6c1050/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.274488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=24dd998ca5d24575a2269da173be3a29, user_id=8f97703389904a5aa2131fbece6c1050, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.277175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.309864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1015/2062] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:30 2022] PUT /identity/v3/projects/24dd998ca5d24575a2269da173be3a29/users/8f97703389904a5aa2131fbece6c1050/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:30.312319 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.312806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.313082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.343935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] MFA Rules not processed for user `8f97703389904a5aa2131fbece6c1050`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:30.345289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:30.355133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:30.429406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1048/2063] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:30.433719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.509963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec607-c594-498a-b181-7ba59b6c3cba admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.523663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.524649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.525138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.525577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.526383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.528787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.571907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1016/2064] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:30 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:30.576174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.606486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4446e-2b35-4bbb-a7b2-dd6b223b0701 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.718310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.719894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.720336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.720831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.721642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.725638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.764675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1049/2065] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:30 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:30.766522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.776920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed7e43-6e72-4ea6-9596-49df17a55749 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.781784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.782377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.782723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.783054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.783792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.785430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.816310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1017/2066] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:30.817012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.828036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a4d1-a2bd-421f-8a6e-b82b8a73514b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.833460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.834236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.834588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.834933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] PATH_INFO: `/v3/projects/dbbd76358434431788887fb32ee83358/users/d61f71187c584e2d8f28b17bf8100e6f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.852061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbd76358434431788887fb32ee83358, user_id=d61f71187c584e2d8f28b17bf8100e6f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.854834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.875550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1050/2067] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:30 2022] PUT /identity/v3/projects/dbbd76358434431788887fb32ee83358/users/d61f71187c584e2d8f28b17bf8100e6f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:30.880536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.890610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb916fa-48af-41dd-9329-8530da28065d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.907083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:30.951515 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.952139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.952421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.952820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.953484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.955087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf50b62-94db-48e8-9588-9b6d8fcd4d70 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:30.965099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1018/2068] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:30.969649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:30.980732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20984ac-efff-466d-a80c-cfcac3dfef5f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:30.986052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:30.986737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:30.987012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:30.987282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] PATH_INFO: `/v3/projects/dbbd76358434431788887fb32ee83358/users/d61f71187c584e2d8f28b17bf8100e6f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:30.989793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbd76358434431788887fb32ee83358, user_id=d61f71187c584e2d8f28b17bf8100e6f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:30.992577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:31.003155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1051/2069] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:30 2022] PUT /identity/v3/projects/dbbd76358434431788887fb32ee83358/users/d61f71187c584e2d8f28b17bf8100e6f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:31.013821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:31.014125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:31.014398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:31.060632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] MFA Rules not processed for user `d61f71187c584e2d8f28b17bf8100e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:31.062692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:31.152593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1019/2070] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:31.166489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:31.168558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:31.253532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a4e5-90f1-4b12-b592-c74e7f3197db admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:31.257939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:31.258642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:31.258991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:31.259463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:31.273514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:31.275130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b00c-07eb-484e-9ece-a4c41f13c639 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:31.292038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1052/2071] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:31.342279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:31.345400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:31.360008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a498d2-ad20-4c24-9fcd-3c031ce9e2af None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:31.371094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:31.371094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:31.371094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:31.371669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:31.437022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:31.438493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2fe00-18e8-4561-a79e-53645707283b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:31.454354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1020/2072] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:33.005302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.005783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.005880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.007462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1053/2073] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:17:33.036553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.037173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.037437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.065616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:33.065616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:33.111818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:33.115830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:33.128626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:33.136433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:33.246591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1021/2074] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:33.259010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35720d07-35cd-4a60-ada2-fa9bd751e54b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:33.260139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:33.262766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:33.273689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:33.274363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.274620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.275055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] PATH_INFO: `/v3/users/d39e2acbd72a42758f63535bac526248` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.276626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39e2acbd72a42758f63535bac526248)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:33.279979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:33.285645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282af5b-76a5-4fff-8cc4-c810c166b472 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:33.299495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:33.299495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.299495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.299495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.309688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:33.311387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:33.323905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1022/2075] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:33.380005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39e2acbd72a42758f63535bac526248'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.389945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39e2acbd72a42758f63535bac526248'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.406716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39e2acbd72a42758f63535bac526248'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.416901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39e2acbd72a42758f63535bac526248'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.417649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3d971dc3-5a85-4991-a1ef-93735c93c6bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39e2acbd72a42758f63535bac526248'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.424695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1054/2076] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:33 2022] DELETE /identity/v3/users/d39e2acbd72a42758f63535bac526248 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:33.607492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:33.619012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4023-4665-4f2e-a7d3-3249071e46e2 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:33.678733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:33.679378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.679673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.679970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] PATH_INFO: `/v3/projects/f186d5bdc9b84b4aaae07b15f7cf132f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.681845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f186d5bdc9b84b4aaae07b15f7cf132f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:33.683844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:33.739792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f186d5bdc9b84b4aaae07b15f7cf132f', 'request_id': 'req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.745333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f186d5bdc9b84b4aaae07b15f7cf132f', 'request_id': 'req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.751538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f186d5bdc9b84b4aaae07b15f7cf132f', 'request_id': 'req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.767086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] The token cache is being invalidate because project f186d5bdc9b84b4aaae07b15f7cf132f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:33.767374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.768043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1d7fb0eb-b21f-41c2-83d5-8c711a951b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:33.769832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1023/2077] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:33 2022] DELETE /identity/v3/projects/f186d5bdc9b84b4aaae07b15f7cf132f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:33.775502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.775787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.776056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.796933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:33.798134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:33.861764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1055/2078] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:33.869989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.870282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.870552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.892533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:33.893768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:33.903730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1024/2079] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:33.908198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:33.918650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1458abf-c99e-4401-b31d-5addef77c5e6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:33.923989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:33.924680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.924967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.925244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.925902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:33.926193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:33.928032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:33.941183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1056/2080] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:33.945721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:33.955442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122021d9-ad32-4303-ad4f-95d347025361 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:33.960925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:33.961526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:33.961804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:33.962073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:33.962706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:33.964675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:33.998817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1025/2081] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:33 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:34.002796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.013309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd69dd-ff91-4d92-96d0-6792be6890d7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.022557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:34.079297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.079917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.080195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.080463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.081138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.083187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.104860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1057/2082] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:34 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:34.109391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.120908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4169-c2c1-4b10-836c-a45cd1bc0892 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.130429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.131093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.131368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.131635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.132276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.134223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.145023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1026/2083] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:34.149518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.160276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4a8d5-7bd0-4b24-a4b1-66c53a9d30b7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.166277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.166886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.167152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.167425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.168068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.169741 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b7d12-4358-4747-bfe9-bd9ccbed3565 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.181254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1058/2084] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:34.185923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.200805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450ad02-1a93-4615-b810-69e39fb93c8b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.207020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.207868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.208287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.208678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] PATH_INFO: `/v3/projects/0e47be8f2ac14c19ab40e293422c845d/users/38a331e431394e4882f31afd31f16d46/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.222979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:34.227984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e47be8f2ac14c19ab40e293422c845d, user_id=38a331e431394e4882f31afd31f16d46, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.233582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.252221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1027/2085] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:34 2022] PUT /identity/v3/projects/0e47be8f2ac14c19ab40e293422c845d/users/38a331e431394e4882f31afd31f16d46/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:34.257625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.257913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.258189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.278462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:34.279822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:34.369965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1059/2086] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:34.385179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.387965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:34.482135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3064-3f9e-42c6-a219-fed7bd7a6d60 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.486074 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.486695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.486981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.487262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.497517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.498996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7df04-bbed-4dbb-9e23-ce1e596cecdc service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.508781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1028/2087] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:34.697100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:34.700461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:34.711952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab67b54-0453-43fe-b2e4-6336c3dbeedc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:34.716824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:34.717549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.717826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.718100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] PATH_INFO: `/v3/users/64207a6fa10843eba27e4e0b7ad21880` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.719409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64207a6fa10843eba27e4e0b7ad21880)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:34.721429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:34.786623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64207a6fa10843eba27e4e0b7ad21880'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:34.801615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64207a6fa10843eba27e4e0b7ad21880'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:34.822429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64207a6fa10843eba27e4e0b7ad21880'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:34.832442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64207a6fa10843eba27e4e0b7ad21880'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:34.834516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64207a6fa10843eba27e4e0b7ad21880'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:34.845153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1060/2088] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:34 2022] DELETE /identity/v3/users/64207a6fa10843eba27e4e0b7ad21880 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:34.850600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:34.854372 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:34.854680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:34.882643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:34.896884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:35.004544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1029/2089] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:35.013321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:35.014971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:35.020973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:35.088114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66bf4-24b9-4876-b2f8-d7fc7d64841a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:35.103262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:35.103998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:35.104423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:35.104742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:35.115339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:35.117291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:35.129560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1061/2090] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:35.404464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:35.406073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:35.423169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85e709-2e29-4d2d-99b4-853b0dad9e49 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:35.429642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:35.430616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:35.430940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:35.431790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] PATH_INFO: `/v3/projects/8c187a7feef44c37a88027de76d979fd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:35.433516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c187a7feef44c37a88027de76d979fd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:35.435529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:35.510493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c187a7feef44c37a88027de76d979fd', 'request_id': 'req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.519563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c187a7feef44c37a88027de76d979fd', 'request_id': 'req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.528001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c187a7feef44c37a88027de76d979fd', 'request_id': 'req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.549766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] The token cache is being invalidate because project 8c187a7feef44c37a88027de76d979fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:35.550067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.550759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.556523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1030/2091] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:35 2022] DELETE /identity/v3/projects/8c187a7feef44c37a88027de76d979fd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:35.558134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:35.571233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1a01-3f37-426a-bdc6-40614e1690db service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:35.684290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:35.685683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:35.686287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:35.686774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] PATH_INFO: `/v3/users/8f97703389904a5aa2131fbece6c1050` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:35.688340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f97703389904a5aa2131fbece6c1050)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:35.691459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:35.751498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f97703389904a5aa2131fbece6c1050'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.761973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f97703389904a5aa2131fbece6c1050'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.775568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f97703389904a5aa2131fbece6c1050'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.787926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f97703389904a5aa2131fbece6c1050'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.788709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f97703389904a5aa2131fbece6c1050'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:35.796001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1062/2092] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:35 2022] DELETE /identity/v3/users/8f97703389904a5aa2131fbece6c1050 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:36.150494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:36.151947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:36.166001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb22edd-fbc9-42c6-8f81-f2a9cc120d9d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:36.179833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:36.259840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:36.260495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:36.260859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:36.261304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] PATH_INFO: `/v3/projects/24dd998ca5d24575a2269da173be3a29` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:36.262431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24dd998ca5d24575a2269da173be3a29)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:36.264586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:36.342603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24dd998ca5d24575a2269da173be3a29', 'request_id': 'req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.351750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24dd998ca5d24575a2269da173be3a29', 'request_id': 'req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.360141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24dd998ca5d24575a2269da173be3a29', 'request_id': 'req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.379525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] The token cache is being invalidate because project 24dd998ca5d24575a2269da173be3a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:36.380243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.380485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.382608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1031/2093] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:36 2022] DELETE /identity/v3/projects/24dd998ca5d24575a2269da173be3a29 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:36.386537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:36.388001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:36.404300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cbbfc-8368-4c1e-9ca5-f3ebc909e5a6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:36.430355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:36.484550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:36.485686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:36.485972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:36.486238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] PATH_INFO: `/v3/users/d61f71187c584e2d8f28b17bf8100e6f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:36.487570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61f71187c584e2d8f28b17bf8100e6f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:36.489593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:36.609802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61f71187c584e2d8f28b17bf8100e6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.621847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61f71187c584e2d8f28b17bf8100e6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.642186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61f71187c584e2d8f28b17bf8100e6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.650505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61f71187c584e2d8f28b17bf8100e6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.651190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-846bf37c-1b33-46e0-813a-2f061490aa23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61f71187c584e2d8f28b17bf8100e6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:36.659241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1063/2094] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:36 2022] DELETE /identity/v3/users/d61f71187c584e2d8f28b17bf8100e6f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:37.430209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:37.432613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:37.448646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4385c5-5fdc-4b0f-96bf-1f2345a1a85d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:37.533658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:37.534382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:37.534722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:37.535403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] PATH_INFO: `/v3/projects/dbbd76358434431788887fb32ee83358` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:37.536869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbbd76358434431788887fb32ee83358)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:37.540463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:37.614396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:37.616574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbbd76358434431788887fb32ee83358', 'request_id': 'req-c8ddf76a-59aa-4c44-95c8-a18c476c33db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:37.623686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbbd76358434431788887fb32ee83358', 'request_id': 'req-c8ddf76a-59aa-4c44-95c8-a18c476c33db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:37.632426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbbd76358434431788887fb32ee83358', 'request_id': 'req-c8ddf76a-59aa-4c44-95c8-a18c476c33db'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:37.670577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] The token cache is being invalidate because project dbbd76358434431788887fb32ee83358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:37.670974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:37.684242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c8ddf76a-59aa-4c44-95c8-a18c476c33db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:37.687702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1032/2095] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:37 2022] DELETE /identity/v3/projects/dbbd76358434431788887fb32ee83358 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:37.693856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:37.694425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:37.694425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:37.719103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:37.720584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:37.827583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1064/2096] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:37.847246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:37.847542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:37.847822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:37.872222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:37.873585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:37.886161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1033/2097] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:37.887357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:37.899509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b98b4-a920-4c71-8fc8-6f712534324e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:37.910606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:37.911561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:37.912062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:37.912339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:37.913037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:37.913343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:37.915678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:37.955016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1065/2098] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:37.957184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:37.971920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1a9c1-399e-4427-9bf9-a615f31c11dd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:37.992868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:37.994130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:37.994634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:37.995113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:37.995956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:37.999046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.041951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1034/2099] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:37 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:38.046107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.067191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36831c-49ba-4ea9-b16c-c5dcd8230f54 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.121579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:38.166863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.170829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.170829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.170829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.170829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.173115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.209395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1066/2100] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:38 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:38.210832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.234355 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f40b-a6bf-4249-b77b-5df2d14efbc2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.249454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.250162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.250444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.250735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.251397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.254303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.271106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1035/2101] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:38.272475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.285581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f10f5-dd6d-4a19-8a19-3115ec860e7c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.292573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.293467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.293929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.294381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.295288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.297341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.317137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1067/2102] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:38.322218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.335481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48e2b-cdbc-49e0-939f-eb58c1c0faed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.348356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.350781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:38.379591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.380983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.381269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.381589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] PATH_INFO: `/v3/projects/c676d84dd791445587bb855511f14118/users/c237992d570941eca99e0b76ea52bfeb/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.391550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37693501-8ab6-4292-9534-e20d4860c0a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.397446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.398053 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.398211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.398581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] PATH_INFO: `/v3/users/a1ec3edb25054c45a293b3fc13b6c6de` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.400547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ec3edb25054c45a293b3fc13b6c6de)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.402215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.411967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c676d84dd791445587bb855511f14118, user_id=c237992d570941eca99e0b76ea52bfeb, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.415589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8636-b5ce-47d2-a5f7-ffca5b92fea0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.441956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1036/2103] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:38 2022] PUT /identity/v3/projects/c676d84dd791445587bb855511f14118/users/c237992d570941eca99e0b76ea52bfeb/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:38.453465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.453680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.453918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.484482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ec3edb25054c45a293b3fc13b6c6de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:38.492425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] MFA Rules not processed for user `c237992d570941eca99e0b76ea52bfeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:38.494512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:38.499928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ec3edb25054c45a293b3fc13b6c6de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:38.531178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ec3edb25054c45a293b3fc13b6c6de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:38.546049 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ec3edb25054c45a293b3fc13b6c6de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:38.546800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ec3edb25054c45a293b3fc13b6c6de'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:38.556912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1068/2104] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:38 2022] DELETE /identity/v3/users/a1ec3edb25054c45a293b3fc13b6c6de => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:38.599911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1037/2105] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:38.603475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.606988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:38.719788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3da67-b53c-4320-abef-61e37bdc272e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.729583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.732966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.734097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.735160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.737245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.740920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.781819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.783366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:38.816671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1069/2106] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:38 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:38.818217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:38.829338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:38.925890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47114cce-838f-40cf-8433-9d59271a7e27 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.931985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.932657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.933178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.933535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78790e9c-e458-464f-b904-ac9396fd93ad admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:38.933656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] PATH_INFO: `/v3/projects/411ef82f869a4b149cbda770debd158e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.934986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=411ef82f869a4b149cbda770debd158e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.937443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.941070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:38.943922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:38.944859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:38.945611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:38.946020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:38.946820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:38.952144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:38.987109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1070/2107] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:38 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:38.988540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.000360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0873657-bf79-4b47-9c8c-a081f5c623e4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.015509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411ef82f869a4b149cbda770debd158e', 'request_id': 'req-7e6431c6-6259-4d63-b769-b865a744b217'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.025786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411ef82f869a4b149cbda770debd158e', 'request_id': 'req-7e6431c6-6259-4d63-b769-b865a744b217'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.035956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411ef82f869a4b149cbda770debd158e', 'request_id': 'req-7e6431c6-6259-4d63-b769-b865a744b217'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.056866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] The token cache is being invalidate because project 411ef82f869a4b149cbda770debd158e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:39.057207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.058591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.064824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1038/2108] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:38 2022] DELETE /identity/v3/projects/411ef82f869a4b149cbda770debd158e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:39.068483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.070494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.112631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.113207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.113488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.113800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.114652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:39.116531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:39.133342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1071/2109] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:39.134314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.136682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.151498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edf395-b604-4a14-acb2-084c68fda26f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.167114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.167892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.168173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.168478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] PATH_INFO: `/v3/projects/694cc18bfdff4a1e9461421cb834ed87/users/1b6d14103a4d4e4989bebeb577c1064d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.196687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6431c6-6259-4d63-b769-b865a744b217 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.202952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.204081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.204619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.204989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] PATH_INFO: `/v3/users/1dcc63dd659b44a98ee0e01de7e458c7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.206600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dcc63dd659b44a98ee0e01de7e458c7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:39.209995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:39.211757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694cc18bfdff4a1e9461421cb834ed87, user_id=1b6d14103a4d4e4989bebeb577c1064d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:39.224632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:39.254164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1072/2110] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:39 2022] PUT /identity/v3/projects/694cc18bfdff4a1e9461421cb834ed87/users/1b6d14103a4d4e4989bebeb577c1064d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:39.258476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.289594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dcc63dd659b44a98ee0e01de7e458c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.315400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dcc63dd659b44a98ee0e01de7e458c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.316919 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dcc63dd659b44a98ee0e01de7e458c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.328254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dcc63dd659b44a98ee0e01de7e458c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.329102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dcc63dd659b44a98ee0e01de7e458c7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:39.351793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1039/2111] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:39 2022] DELETE /identity/v3/users/1dcc63dd659b44a98ee0e01de7e458c7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:39.372467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de329-ecef-462d-8e59-8b4d23bedb32 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.386648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:39.462833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.464021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.464526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.465548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.467012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:39.469177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3a916-0da2-4a58-a031-d4f7531ff364 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:39.485024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1073/2112] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:39.489695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.491428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.507432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f616-52ca-4479-9970-d768c2cf008a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.514286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.514286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.514286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.514286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] PATH_INFO: `/v3/projects/694cc18bfdff4a1e9461421cb834ed87/users/1b6d14103a4d4e4989bebeb577c1064d/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.516065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694cc18bfdff4a1e9461421cb834ed87, user_id=1b6d14103a4d4e4989bebeb577c1064d, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:39.519122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:39.529603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:39.537033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1040/2113] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:39 2022] PUT /identity/v3/projects/694cc18bfdff4a1e9461421cb834ed87/users/1b6d14103a4d4e4989bebeb577c1064d/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:39.550103 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.550387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.550663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.597842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] MFA Rules not processed for user `1b6d14103a4d4e4989bebeb577c1064d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:39.599855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.711912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1074/2114] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:39.721972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.724476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.811927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:39.816287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.868266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3f85-078e-4601-b78c-25424fe97d78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.880234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.884277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.884663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.884939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:39.887205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:39.983497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fcae0-ffa6-45d9-ba20-f3c2e4cf977e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:39.995959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:39.996747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:39.997275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:39.998498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] PATH_INFO: `/v3/projects/3efff253557c45d295e2381e0a9b4454` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.004450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efff253557c45d295e2381e0a9b4454)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.004450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.024077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.025835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864a550-e85a-41f3-98a7-824f4960ae00 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.043346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1041/2115] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:40.093378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.093698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.094049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.105109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:40.116123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efff253557c45d295e2381e0a9b4454', 'request_id': 'req-a4b517df-39a4-4546-8891-e45c2e07cc5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:40.134593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] MFA Rules not processed for user `38a331e431394e4882f31afd31f16d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:40.137126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:40.137506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efff253557c45d295e2381e0a9b4454', 'request_id': 'req-a4b517df-39a4-4546-8891-e45c2e07cc5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:40.147774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efff253557c45d295e2381e0a9b4454', 'request_id': 'req-a4b517df-39a4-4546-8891-e45c2e07cc5a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:40.162515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] The token cache is being invalidate because project 3efff253557c45d295e2381e0a9b4454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:40.163061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:40.164545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4b517df-39a4-4546-8891-e45c2e07cc5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:40.167699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1075/2116] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:39 2022] DELETE /identity/v3/projects/3efff253557c45d295e2381e0a9b4454 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:17:40.171625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c96a2-ba30-4850-8847-9629d622401e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.171915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c96a2-ba30-4850-8847-9629d622401e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.172186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c96a2-ba30-4850-8847-9629d622401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.197954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-925c96a2-ba30-4850-8847-9629d622401e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:40.199476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-925c96a2-ba30-4850-8847-9629d622401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:40.261992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1042/2117] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:40.277544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.278719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1076/2118] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:40.279030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:40.286093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.286379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.286655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.311261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:40.312222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:40.331682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1077/2119] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:40.333220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.336831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:40.343870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8e108-d084-466a-937d-b874474fe6b2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.349540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.350274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.350640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.352036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.352036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:40.352247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.354061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.374490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1078/2120] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:40.383629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0cf5a-7557-4452-8af8-3ac18ba06605 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.387333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.394652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.395617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.395895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.396200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.404225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496572e2-a81c-4ad2-8204-eb97e15df389 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.405008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.412352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.413809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.415246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.415659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.416096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.416761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.419036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.427068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1043/2121] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:40.467830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1079/2122] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:40.472118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.490632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc288-048b-4acb-be7e-06606f29060f service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.591773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.592588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.592949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.593369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.594218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.597222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.633979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1044/2123] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:40.637496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.650143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb79ccc-7a7d-4bf1-9d50-163bfa270c59 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.671650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.671922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.672167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.672437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.673178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.677062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.694694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1080/2124] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:40.695963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.714539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796ca11-f07e-4b5a-a753-f47d3449e66a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.725693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.726591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.726956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.727291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.728565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.730619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd520-f63c-40a7-9b07-f667c7608aa1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.748908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1045/2125] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:40.752081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.763690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b260a1-95c6-46d3-9e9f-6d52334bb824 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:40.779015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:40.780805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.781207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.781490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] PATH_INFO: `/v3/projects/2f950e927e834249928e22eb9c65deac/users/3a56eabd1c38426fbfa70fa74d655524/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.807362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f950e927e834249928e22eb9c65deac, user_id=3a56eabd1c38426fbfa70fa74d655524, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:40.810879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:40.828076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1081/2126] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:40 2022] PUT /identity/v3/projects/2f950e927e834249928e22eb9c65deac/users/3a56eabd1c38426fbfa70fa74d655524/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:40.839863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256142a-9e02-404b-b4ee-c46230eb6e93 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:40.840076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256142a-9e02-404b-b4ee-c46230eb6e93 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:40.840440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256142a-9e02-404b-b4ee-c46230eb6e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:40.878173 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3256142a-9e02-404b-b4ee-c46230eb6e93 None None] MFA Rules not processed for user `3a56eabd1c38426fbfa70fa74d655524`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:40.879507 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3256142a-9e02-404b-b4ee-c46230eb6e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:40.971321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1046/2127] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:40.987063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:40.989224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:41.123816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ce20a-0d00-4ea4-b2d5-c9ef640ae00c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:41.131455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:41.132236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:41.132689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:41.132975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:41.140389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:41.145463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:41.149833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:41.166464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1082/2128] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:42.292829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197638-2c88-4a5b-95b0-3e905bfd663f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:42.293325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197638-2c88-4a5b-95b0-3e905bfd663f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:42.293429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197638-2c88-4a5b-95b0-3e905bfd663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:42.312724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2197638-2c88-4a5b-95b0-3e905bfd663f None None] MFA Rules not processed for user `38a331e431394e4882f31afd31f16d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:42.313912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2197638-2c88-4a5b-95b0-3e905bfd663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:42.346442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:42.372433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1047/2129] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:51.157083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:52.364829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=9 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:53.554073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:53.555624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:53.651903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d8900-cf9e-4308-ae92-1474242f5b78 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:53.656980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:53.657629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:53.657936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:53.658185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] PATH_INFO: `/v3/users/c237992d570941eca99e0b76ea52bfeb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:53.659309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c237992d570941eca99e0b76ea52bfeb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:53.661310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:53.711595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c237992d570941eca99e0b76ea52bfeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:53.721442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c237992d570941eca99e0b76ea52bfeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:53.740705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c237992d570941eca99e0b76ea52bfeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:53.752744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c237992d570941eca99e0b76ea52bfeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:53.753439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c237992d570941eca99e0b76ea52bfeb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:53.760551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1083/2130] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:53 2022] DELETE /identity/v3/users/c237992d570941eca99e0b76ea52bfeb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:53.765724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:53.766212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:53.766301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:53.787473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:53.789364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:53.854540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1048/2131] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:53.864559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:53.866216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:53.927374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73844951-7439-4d55-8d18-96280a26bc29 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:53.933207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:53.933901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:53.934251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:53.934592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:53.954042 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:53.955392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:53.968204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1084/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:54.435694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:54.437323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:54.450067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb447d-9034-4494-b0d1-1d2516303ac4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:54.456233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:54.457214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:54.457639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:54.458067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] PATH_INFO: `/v3/projects/c676d84dd791445587bb855511f14118` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:54.465201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c676d84dd791445587bb855511f14118)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:54.467164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:54.531960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c676d84dd791445587bb855511f14118', 'request_id': 'req-b993bf19-2dee-4451-aac8-58aa4924f992'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.537595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c676d84dd791445587bb855511f14118', 'request_id': 'req-b993bf19-2dee-4451-aac8-58aa4924f992'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.544106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c676d84dd791445587bb855511f14118', 'request_id': 'req-b993bf19-2dee-4451-aac8-58aa4924f992'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.561298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] The token cache is being invalidate because project c676d84dd791445587bb855511f14118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:54.561587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.562211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.563790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1049/2133] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:54 2022] DELETE /identity/v3/projects/c676d84dd791445587bb855511f14118 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:54.567986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:54.578101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3cfa6-d948-438c-a8cf-9a55ea30cfd4 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:54.611520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:54.658189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:54.659039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:54.659325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:54.660050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] PATH_INFO: `/v3/users/1b6d14103a4d4e4989bebeb577c1064d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:54.661315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6d14103a4d4e4989bebeb577c1064d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:54.663468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:54.711961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6d14103a4d4e4989bebeb577c1064d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.718826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6d14103a4d4e4989bebeb577c1064d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.732131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6d14103a4d4e4989bebeb577c1064d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.742143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6d14103a4d4e4989bebeb577c1064d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.742915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-27f766f8-5d55-4b95-9ee3-b10713ab2ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6d14103a4d4e4989bebeb577c1064d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:54.749896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1085/2134] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:54 2022] DELETE /identity/v3/users/1b6d14103a4d4e4989bebeb577c1064d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:55.184473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.186044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:55.204240 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993bf19-2dee-4451-aac8-58aa4924f992 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.280986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.281669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.281952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.282230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] PATH_INFO: `/v3/projects/694cc18bfdff4a1e9461421cb834ed87` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.283557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694cc18bfdff4a1e9461421cb834ed87)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.285609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.327058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:55.350394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694cc18bfdff4a1e9461421cb834ed87', 'request_id': 'req-e37422ae-16b0-4b49-a182-2e900fd60995'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:55.356279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694cc18bfdff4a1e9461421cb834ed87', 'request_id': 'req-e37422ae-16b0-4b49-a182-2e900fd60995'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:55.362812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694cc18bfdff4a1e9461421cb834ed87', 'request_id': 'req-e37422ae-16b0-4b49-a182-2e900fd60995'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:55.377745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] The token cache is being invalidate because project 694cc18bfdff4a1e9461421cb834ed87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:55.378034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:55.378652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e37422ae-16b0-4b49-a182-2e900fd60995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:55.380203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1050/2135] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:55 2022] DELETE /identity/v3/projects/694cc18bfdff4a1e9461421cb834ed87 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:55.385143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.385458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.385750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.408066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:55.409973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:55.475296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1086/2136] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:55.483393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.483708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.483993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.504604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:55.505831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:55.515131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1051/2137] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:55.519532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.530117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ea562-fe93-44a1-974c-6eaa1fb24517 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.535945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.536596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.537092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.537521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.538198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:55.538500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.540460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.556218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1087/2138] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:55.557697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.567463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a2914-8ff0-40b7-a90b-969022ae0dcb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.575183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.575799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.576072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.576335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.577080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.579180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.615126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1052/2139] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:55 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:55.616408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.627035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a228195-22ea-4e35-b468-5f77ff398334 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.655022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:55.689846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.690570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.690853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.691168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.691742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.693819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.717094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1088/2140] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:55 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:55.721224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.732860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5859-4fb9-44fb-a8a3-71f65e64c05c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.737479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.738079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.738359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.738627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.739280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.740935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.751249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1053/2141] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:55.755349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.765788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43dd2-0a22-4b2d-acb0-c843bba09932 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.771703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.772309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.772613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.773525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.773694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.779132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc96d1-5403-42e8-8c9e-d36db28c94e3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.794649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1089/2142] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:55.799316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.809995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914101-2af9-4286-b7e1-ca3d0378fadf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:55.815398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:55.816052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.816332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.816640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] PATH_INFO: `/v3/projects/c8a63438f9e0494d8f18e899b4157dd4/users/28f5f4c5086841ac86fb4509dbc3d239/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.832051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a63438f9e0494d8f18e899b4157dd4, user_id=28f5f4c5086841ac86fb4509dbc3d239, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:55.834707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:55.850799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1054/2143] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:55 2022] PUT /identity/v3/projects/c8a63438f9e0494d8f18e899b4157dd4/users/28f5f4c5086841ac86fb4509dbc3d239/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:55.861655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:55.861964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:55.862244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:55.892303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] MFA Rules not processed for user `28f5f4c5086841ac86fb4509dbc3d239`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:55.893592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:55.957923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1090/2144] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:55.969087 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:55.971080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:56.033890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97cfa2-bbda-4944-8278-2339ce42ed6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:56.038256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:56.038859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:56.039482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:56.039724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:56.048931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:56.050359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec394aa-29a8-49d2-8c3d-95c19004c88d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:56.059198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1055/2145] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:56.115447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:56.116896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:56.186102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6dba-c5bb-4c35-8294-4a736aabe3ee None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:56.192010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:56.192814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:56.193217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:56.193525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] PATH_INFO: `/v3/users/3a56eabd1c38426fbfa70fa74d655524` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:56.194660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a56eabd1c38426fbfa70fa74d655524)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:56.196622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:56.243853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:56.250556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a56eabd1c38426fbfa70fa74d655524'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:56.261072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a56eabd1c38426fbfa70fa74d655524'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:56.274917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a56eabd1c38426fbfa70fa74d655524'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:56.282779 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a56eabd1c38426fbfa70fa74d655524'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:56.283571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a56eabd1c38426fbfa70fa74d655524'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:56.290615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1091/2146] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:56 2022] DELETE /identity/v3/users/3a56eabd1c38426fbfa70fa74d655524 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:56.295717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:56.296242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:56.296242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:56.316759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:56.317988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:56.379967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:56.395155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1056/2147] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:56.406635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:56.408065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:56.472277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c9536-ccb7-4a21-b916-17eff6722cce admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:56.476056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:56.476776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:56.477066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:56.477337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:56.486860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:56.488447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6479-7d8e-40e9-9216-9285422d60af service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:56.498292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1092/2148] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:57.049135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.050599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:57.062709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f164d-6bb0-4cc9-8d35-9605ad2a9617 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.068111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.068794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.069045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.069318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] PATH_INFO: `/v3/projects/2f950e927e834249928e22eb9c65deac` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.070659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f950e927e834249928e22eb9c65deac)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.072626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.122549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f950e927e834249928e22eb9c65deac', 'request_id': 'req-70098f5c-7e9b-4e53-a64a-42c6d726e282'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.128765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f950e927e834249928e22eb9c65deac', 'request_id': 'req-70098f5c-7e9b-4e53-a64a-42c6d726e282'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.134987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f950e927e834249928e22eb9c65deac', 'request_id': 'req-70098f5c-7e9b-4e53-a64a-42c6d726e282'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.148890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] The token cache is being invalidate because project 2f950e927e834249928e22eb9c65deac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:57.149181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.149804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-70098f5c-7e9b-4e53-a64a-42c6d726e282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.151369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1057/2149] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:57 2022] DELETE /identity/v3/projects/2f950e927e834249928e22eb9c65deac => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:57.157036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.157503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.157606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.178286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:57.179498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:57.241622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1093/2150] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:57.249945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.250238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.250510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.270758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:57.271979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:57.280146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1058/2151] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:57.284365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.294716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade17375-e62c-4b07-83ae-0f42b544041e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.299618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.300206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.300475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.300766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.301393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:57.301688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.303286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.315365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1094/2152] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:17:57.320023 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.335250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6170c9-087d-459b-af02-7600e6c27073 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.344586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.345185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.345461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.345731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.346373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.348278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.379434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1059/2153] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:57 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:17:57.383376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.393420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b4f-2595-4c26-99ab-eac3d124f06e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.437115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:57.450998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.451604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.451878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.452144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.452806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.454739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.475999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1095/2154] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:57 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:57.480154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.490719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a36f8c-f40a-41b5-b150-8edecaaf0b1e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.495596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.496214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.496483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.496779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.497423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.499001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.509541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1060/2155] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:57.513686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.525095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe462c-8e3a-4fea-b00a-a6f26038b041 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.530478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.531079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.531357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.531627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.532278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.533959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7d30a-83ed-4e2e-8884-7735f338d123 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.548209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1096/2156] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:57.557579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.567516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f109a7c-acd6-4c00-a0c6-abe6a6463bdd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.572413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.573099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.573378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.573684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] PATH_INFO: `/v3/projects/da5a30fbaf8f4f11b6c1d3b0ccaa7186/users/e89922da1c7a4a7c90d898d08721e65a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.593852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5a30fbaf8f4f11b6c1d3b0ccaa7186, user_id=e89922da1c7a4a7c90d898d08721e65a, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.596710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.609682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1061/2157] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:57 2022] PUT /identity/v3/projects/da5a30fbaf8f4f11b6c1d3b0ccaa7186/users/e89922da1c7a4a7c90d898d08721e65a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:57.621242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.621634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.621969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.649919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] MFA Rules not processed for user `e89922da1c7a4a7c90d898d08721e65a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:57.651130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:57.722490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1097/2158] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:57.736642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.738477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:57.801455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2683-e2f0-46cb-acd8-f5c257969e45 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.805244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.805856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.806131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.806399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.814766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.816482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fa442-b6f1-4ceb-a2f9-be08e106c7e1 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.834134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1062/2159] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:17:57.860683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:57.923689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395d89f-c6e2-476d-9ea4-603c292fc223 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:57.928762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:57.929440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:57.929704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:57.929972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] PATH_INFO: `/v3/users/e89922da1c7a4a7c90d898d08721e65a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:57.931132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89922da1c7a4a7c90d898d08721e65a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:57.933131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:57.983910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89922da1c7a4a7c90d898d08721e65a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.991868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89922da1c7a4a7c90d898d08721e65a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:57.993063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:58.008444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89922da1c7a4a7c90d898d08721e65a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.017705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89922da1c7a4a7c90d898d08721e65a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.018634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89922da1c7a4a7c90d898d08721e65a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.024996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1098/2160] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:17:57 2022] DELETE /identity/v3/users/e89922da1c7a4a7c90d898d08721e65a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:58.029671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.029972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.030233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.050583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:58.051851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:58.061238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:58.123768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1063/2161] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:17:58.137262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:58.140613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:58.208404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983bd3-e7cf-49df-979c-521403d88a71 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:58.212181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:58.212819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.213095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.213356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.222782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:58.224224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0220b07-99a3-423e-9e00-ab2ed27ace27 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:58.234289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1099/2162] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:17:58.511592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:58.513481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:58.527936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac99ed-2f3e-4242-a543-20f6ca11e9fc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:58.533588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:58.534219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.534501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.534772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] PATH_INFO: `/v3/projects/da5a30fbaf8f4f11b6c1d3b0ccaa7186` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.535908 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5a30fbaf8f4f11b6c1d3b0ccaa7186)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:58.537845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:58.595830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5a30fbaf8f4f11b6c1d3b0ccaa7186', 'request_id': 'req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.601378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5a30fbaf8f4f11b6c1d3b0ccaa7186', 'request_id': 'req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.607581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5a30fbaf8f4f11b6c1d3b0ccaa7186', 'request_id': 'req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.622843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] The token cache is being invalidate because project da5a30fbaf8f4f11b6c1d3b0ccaa7186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:17:58.623136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.623765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a887bac3-ebce-4ed0-8d48-b4e3a34d6fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:17:58.625315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1064/2163] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:17:58 2022] DELETE /identity/v3/projects/da5a30fbaf8f4f11b6c1d3b0ccaa7186 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:58.630813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.631097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.631373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.651181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:58.652385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:58.713456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1100/2164] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:58.721214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.721582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.721750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.746685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:58.747996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:58.757471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1065/2165] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:58.761648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:58.777801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977366-5483-4381-9db3-e5cd3114f401 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:58.782757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:58.783372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.783763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.784153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.785107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:17:58.785422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:58.787242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:58.804582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1101/2166] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:17:58.805353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:58.816085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ba44-6b90-4a73-b53a-a1f9efe2e04f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:58.821996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:58.822800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.822980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.823260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.824084 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:58.839299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:58.872914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1066/2167] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:17:58 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:58.877169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:58.889023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467f07d-d65d-4c57-96c8-3e58efb5df27 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:58.963945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:58.964604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:58.964886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:58.965158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:58.965792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:58.967725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:58.977632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:58.996316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1102/2168] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:17:58 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:17:58.997162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:59.007889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc56b0-ebbc-4964-b398-699efa713dd2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:59.012480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:59.013119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:59.013398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:59.013667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:59.014321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:59.015920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:59.026376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1067/2169] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:17:59.030686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:59.042097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819ad0-3e50-41e3-b7c9-4d495ed2ec24 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:59.046901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:59.047515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:59.047796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:59.048147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:59.048834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:59.050445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdae59-eae4-4adc-88d6-18b4a4d60b4c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:59.061506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1103/2170] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:17:59.065660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:59.076117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d73f59-db4c-4d81-9b74-6908766f1b55 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:59.081289 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:59.081950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:59.082227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:59.082496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] PATH_INFO: `/v3/projects/71d342bcc8534253b0da2c669f023a28/users/8d1c6271efcb43758e454ae8399a9b8e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:59.099891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d342bcc8534253b0da2c669f023a28, user_id=8d1c6271efcb43758e454ae8399a9b8e, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:59.102928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:59.120426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1068/2171] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:17:59 2022] PUT /identity/v3/projects/71d342bcc8534253b0da2c669f023a28/users/8d1c6271efcb43758e454ae8399a9b8e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:17:59.132430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:59.132869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:59.133175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:59.167746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] MFA Rules not processed for user `8d1c6271efcb43758e454ae8399a9b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:17:59.169002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:59.238388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1104/2172] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:17:59.251474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:17:59.253478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:17:59.303218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:17:59.334637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d677e9-6787-4991-a26d-ed56e10276b0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:17:59.348719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:17:59.349356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:17:59.349631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:17:59.349898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:17:59.365277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:17:59.368180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d18cd-dcf9-4b6f-8635-b4a68de90aa3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:17:59.381979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1069/2173] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:17:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:00.223650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:00.225270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:00.293615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b3cc1-3bfb-4ae7-9ba6-677b4487a5e0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:00.298629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:00.299282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:00.299565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:00.299832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] PATH_INFO: `/v3/users/28f5f4c5086841ac86fb4509dbc3d239` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:00.301038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f5f4c5086841ac86fb4509dbc3d239)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:00.302971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:00.357832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f5f4c5086841ac86fb4509dbc3d239'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:00.364558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f5f4c5086841ac86fb4509dbc3d239'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:00.377255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f5f4c5086841ac86fb4509dbc3d239'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:00.378978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:00.387160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f5f4c5086841ac86fb4509dbc3d239'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:00.388301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f5f4c5086841ac86fb4509dbc3d239'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:00.394834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1105/2174] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:00 2022] DELETE /identity/v3/users/28f5f4c5086841ac86fb4509dbc3d239 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:00.399380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:00.399854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:00.399943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:00.419474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:00.420710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:00.484932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1070/2175] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:00.494942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:00.496462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:00.555100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2872c60-602c-4850-a6ac-fa1e0a187a68 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:00.559377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:00.559980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:00.560254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:00.560644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:00.570233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:00.571662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a31a3-d8e7-4f43-b398-bb0a62a9cc07 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:00.592606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1106/2176] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:00.921458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:00.923179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:00.940183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1e9ab-d5ab-4601-8a1c-5aded1fd8bf8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:00.948735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:00.948735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:00.948735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:00.948735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] PATH_INFO: `/v3/projects/c8a63438f9e0494d8f18e899b4157dd4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:00.949594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a63438f9e0494d8f18e899b4157dd4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:00.951803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.003547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a63438f9e0494d8f18e899b4157dd4', 'request_id': 'req-b701aeac-df7f-4f8d-a415-413e136bd629'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:01.009638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a63438f9e0494d8f18e899b4157dd4', 'request_id': 'req-b701aeac-df7f-4f8d-a415-413e136bd629'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:01.015847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a63438f9e0494d8f18e899b4157dd4', 'request_id': 'req-b701aeac-df7f-4f8d-a415-413e136bd629'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:01.032759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] The token cache is being invalidate because project c8a63438f9e0494d8f18e899b4157dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:01.033053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:01.033677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b701aeac-df7f-4f8d-a415-413e136bd629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:01.035202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1071/2177] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:00 2022] DELETE /identity/v3/projects/c8a63438f9e0494d8f18e899b4157dd4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:01.040458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508664-776c-445c-8379-a3b89f692875 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.040774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508664-776c-445c-8379-a3b89f692875 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.041076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508664-776c-445c-8379-a3b89f692875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.060863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-81508664-776c-445c-8379-a3b89f692875 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:01.062073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81508664-776c-445c-8379-a3b89f692875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:01.122892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1107/2178] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:01.193201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.193488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.193759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.214596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:01.215992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:01.224319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1072/2179] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:01.228786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508664-776c-445c-8379-a3b89f692875 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.238715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508664-776c-445c-8379-a3b89f692875 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.243440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.244061 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.244602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.245096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.246288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:18:01.246288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.247617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.259643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1108/2180] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:18:01.264277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.273911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dcb69-0ab2-40e8-881f-5ca146f631fa None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.278740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.279345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.279624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.279899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.280568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.282474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.311816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1073/2181] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:18:01 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:01.315957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.330391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa123092-2680-441a-a06e-b89beef42a28 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.392121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.392874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.393170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.393429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.394091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.396009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.423310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1109/2182] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:18:01 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:01.424950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.435815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33d05-3189-4aa4-9cab-48ca9c8da0d4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.441385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.442008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.442292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.442563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.443201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.444977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.455309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1074/2183] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:01.459762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.470436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9a77f-8124-4942-82c5-fe0f0719e3ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.475341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.475950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.476243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.476614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.477174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.478798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd030d-d25e-4cf1-a2a1-74786b9a5128 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.484425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:01.494781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.495358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1110/2184] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:18:01.510485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8eb5-f5a4-47b2-b68f-08933af8d18c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.515617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.516338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.516693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.516970 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] PATH_INFO: `/v3/projects/f66f956fcf914783be2e494b885f3348/users/5bddf7fd171842878e359beebbf57a67/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.534758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66f956fcf914783be2e494b885f3348, user_id=5bddf7fd171842878e359beebbf57a67, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.538423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.548661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:01.555049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1075/2185] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:01 2022] PUT /identity/v3/projects/f66f956fcf914783be2e494b885f3348/users/5bddf7fd171842878e359beebbf57a67/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:18:01.566064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.566466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.566799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.594521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] MFA Rules not processed for user `5bddf7fd171842878e359beebbf57a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:01.595790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:01.659573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1111/2186] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:01.671708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:01.673776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:01.735323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568bae6-fd89-4c95-a4fb-84f044a216f9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:01.739750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:01.740365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:01.740693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:01.740973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:01.752298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:01.753800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1044f3b-6e0a-4181-9b58-4e9be49fc21d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:01.763322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1076/2187] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:11.656141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=32 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:11.758956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=27 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:16.227790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:16.229763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:16.308008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c42c2-6f4f-4fd7-9f4a-bb024c8c809f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:16.313009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:16.314266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:16.314545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:16.314814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] PATH_INFO: `/v3/users/acadef019281457dbdb757118a129f2d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:16.316078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] RBAC: Authorizing `identity:delete_user(user_id=acadef019281457dbdb757118a129f2d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:16.318182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:16.378111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acadef019281457dbdb757118a129f2d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:16.388285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acadef019281457dbdb757118a129f2d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:16.403481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acadef019281457dbdb757118a129f2d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:16.414915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acadef019281457dbdb757118a129f2d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:16.415670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acadef019281457dbdb757118a129f2d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:16.428787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:16.429298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:16.429592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:16.433531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1112/2188] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:16 2022] DELETE /identity/v3/users/acadef019281457dbdb757118a129f2d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:16.454534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:16.455675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:16.536727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1077/2189] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:16.550725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:16.552205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:16.650806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc66-da83-419c-80df-7e404310defc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:16.656167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:16.657681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:16.657681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:16.657681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:16.677613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:16.680193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:16.701584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1113/2190] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:16.998288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:16.999839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.014750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ec0e7-704b-40d5-baf3-70673cf86744 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.020233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.021055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.021358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.021613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] PATH_INFO: `/v3/users/8d1c6271efcb43758e454ae8399a9b8e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.022891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1c6271efcb43758e454ae8399a9b8e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.024850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.034473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.036663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.061548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8b344-0dd5-4200-875b-2b3f5dfe7b8e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.068000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.068712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.069029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.069248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] PATH_INFO: `/v3/projects/cd98670ba9314aba9a92fe1ef7b26388` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.070442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd98670ba9314aba9a92fe1ef7b26388)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.072408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.091665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:17.095713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1c6271efcb43758e454ae8399a9b8e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.107802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1c6271efcb43758e454ae8399a9b8e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.109206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1c6271efcb43758e454ae8399a9b8e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.121297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1c6271efcb43758e454ae8399a9b8e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.122176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ef4de9a5-af37-40f7-8d25-213d2f7a736b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1c6271efcb43758e454ae8399a9b8e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.134823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd98670ba9314aba9a92fe1ef7b26388', 'request_id': 'req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.144431 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1078/2191] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:16 2022] DELETE /identity/v3/users/8d1c6271efcb43758e454ae8399a9b8e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:17.145332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd98670ba9314aba9a92fe1ef7b26388', 'request_id': 'req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.149032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.149326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.149688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.165142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd98670ba9314aba9a92fe1ef7b26388', 'request_id': 'req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.172465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:17.173851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.211739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] The token cache is being invalidate because project cd98670ba9314aba9a92fe1ef7b26388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:17.212248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.215329 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-331c93d9-9b6e-4562-8e5b-ed6eaa135cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:17.223184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1114/2192] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:17 2022] DELETE /identity/v3/projects/cd98670ba9314aba9a92fe1ef7b26388 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:17.226931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d19-5d22-4503-b008-b1b2828c1888 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.227253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d19-5d22-4503-b008-b1b2828c1888 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.227497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c6d19-5d22-4503-b008-b1b2828c1888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.262602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-669c6d19-5d22-4503-b008-b1b2828c1888 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:17.263963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-669c6d19-5d22-4503-b008-b1b2828c1888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.272766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1079/2193] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:17.280615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.282817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.291994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1115/2194] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:17.293683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.293972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.294261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.304387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:17.333498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:17.334849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:17.354573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1116/2195] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:17.357877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.372992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f5ce-fbcf-4a90-9e83-4f3d0ca5721b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.380217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7129e0-4329-4452-b29a-1739ce1ce10e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.383100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.384012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.384297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.384599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.397936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.399389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.405599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.407481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.408995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.409306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.409996 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:18:17.410409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.414744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.417791 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1080/2196] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:17.439957 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1117/2197] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:18:17.440222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.450503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece19e0-66e6-49b1-a08b-af9bfd200be7 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.464628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.465366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.466251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.466570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.467856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.471338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.505881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1081/2198] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:18:17.510537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.522302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701531fe-ddaa-4882-a5b6-7e13f05f5c13 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.626960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.627684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.628034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.628416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.629242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.631039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.666610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1118/2199] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:17.671041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.684140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681dc16-2416-4731-8a1e-3bd026be6628 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.689634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.690410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.690763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.691103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.691847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.693753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.716188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1082/2200] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:17.717780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.739266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755090-f488-4a34-bfee-3f42e21ee504 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.754485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.754913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.755085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.755378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.756057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.757990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e2f25-8af7-4d28-a65d-8f7699f08ab2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.784332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1119/2201] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:17.789680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.805826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191860c-ce90-4401-99e1-52ef33abc0c6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:17.819478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:17.821249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.821530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.821801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] PATH_INFO: `/v3/projects/6b7d4dd115f647ba8856c2420be2e7e1/users/91e0a548fbb24069afe1bae1107ff3a8/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.889351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7d4dd115f647ba8856c2420be2e7e1, user_id=91e0a548fbb24069afe1bae1107ff3a8, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:17.895205 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:17.931034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1083/2202] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:17 2022] PUT /identity/v3/projects/6b7d4dd115f647ba8856c2420be2e7e1/users/91e0a548fbb24069afe1bae1107ff3a8/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:17.945836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:17.949137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:17.949691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:17.980977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:17.983567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:18.005547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] MFA Rules not processed for user `91e0a548fbb24069afe1bae1107ff3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:18.007379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:18.113879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077d19c-8395-4ca1-90f4-be31cec55e21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.119177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.119941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.120292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.120671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] PATH_INFO: `/v3/projects/71d342bcc8534253b0da2c669f023a28` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.123665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d342bcc8534253b0da2c669f023a28)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.125285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.128840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:18.173157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1120/2203] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:18.174560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:18.203047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d342bcc8534253b0da2c669f023a28', 'request_id': 'req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:18.205600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc18d-e223-4582-b2a1-9dba14103442 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.210116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d342bcc8534253b0da2c669f023a28', 'request_id': 'req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:18.212466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:18.218018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d342bcc8534253b0da2c669f023a28', 'request_id': 'req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:18.241989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] The token cache is being invalidate because project 71d342bcc8534253b0da2c669f023a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:18.241989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:18.241989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:18.241989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1084/2204] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:17 2022] DELETE /identity/v3/projects/71d342bcc8534253b0da2c669f023a28 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:18.373532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.374271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.374463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.374728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.375846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.380879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.454356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1121/2205] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:18:18 2022] POST /identity/v3/projects => generated 373 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:18.457727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:18.481823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:18.509555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d7242-dfa1-407a-afe3-a7b87c4b930c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.571169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.571854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.572147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.572421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.573137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.575487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.623327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1085/2206] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:18:18 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:18.624582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:18.639510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd30d1-42b4-4d67-83a8-6eb03fee2bc9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.644727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.645354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.645631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.645906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.646607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.648234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.665956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1122/2207] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:18.668250 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:18.696989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24528-610b-435c-9eb1-b599bcc6f0c7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.704947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.705581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.706215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.706503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.707195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.725108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98d971-16af-46b6-b8d9-0b2e521b742b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.750057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1086/2208] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:18:18.754991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:18.771095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91152c1-4393-4670-be43-b7ed4370ff4e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:18.782369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:18.783548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.784089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.784666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] PATH_INFO: `/v3/projects/fbceda11ed964612a4a6945c32bde428/users/99e28ada8e9f4b90bbb41a0d00e6a505/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.829178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbceda11ed964612a4a6945c32bde428, user_id=99e28ada8e9f4b90bbb41a0d00e6a505, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:18.848819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:18.881517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1123/2209] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:18 2022] PUT /identity/v3/projects/fbceda11ed964612a4a6945c32bde428/users/99e28ada8e9f4b90bbb41a0d00e6a505/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:18.885423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:18.885830 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:18.886186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:18.935000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] MFA Rules not processed for user `99e28ada8e9f4b90bbb41a0d00e6a505`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:18.936342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.029770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1087/2210] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:19.033616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.119951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14df55-3d20-4b8f-89de-5f2410cb8732 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.125742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.127187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.127471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.128172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.129065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.131530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.179393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1124/2211] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:18:19 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:18:19.181424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.194494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc252cb2-365c-4c86-8592-74ace3d272e0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.254643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:19.263354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.263803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.264105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.264495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.265319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.267953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.296555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1088/2212] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:18:19 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:19.299984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.311826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456dff6-7114-46ec-9a78-e9c6e0c07be5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.318795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.319483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.319832 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.320203 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.320964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.322660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.336063 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1125/2213] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:19.339215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.350374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b1f3b-7048-4f5a-b859-7f5985ab13c4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.355082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.355798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.356133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.356410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] PATH_INFO: `/v3/projects/127fe9a5e23a4ed385b5ce9771760569/users/e6cb7112373542648ed342e8ee5d7a6e/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.372322 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=127fe9a5e23a4ed385b5ce9771760569, user_id=e6cb7112373542648ed342e8ee5d7a6e, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.375029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.388822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1089/2214] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:19 2022] PUT /identity/v3/projects/127fe9a5e23a4ed385b5ce9771760569/users/e6cb7112373542648ed342e8ee5d7a6e/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:18:19.392797 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.402882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755277-17b6-4274-896c-157e14ecb63e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.412429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:19.466134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.466751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.467027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.467297 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.468028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.469678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d29fa9-ce0c-4607-bf84-5f4263bba324 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.480783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1126/2215] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:19.485129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.496849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346462db-0393-4dbb-bd11-798fb4842e7b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.501581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.502262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.502610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.502891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] PATH_INFO: `/v3/projects/127fe9a5e23a4ed385b5ce9771760569/users/e6cb7112373542648ed342e8ee5d7a6e/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.504823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127fe9a5e23a4ed385b5ce9771760569, user_id=e6cb7112373542648ed342e8ee5d7a6e, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.507448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.520876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1090/2216] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:19 2022] PUT /identity/v3/projects/127fe9a5e23a4ed385b5ce9771760569/users/e6cb7112373542648ed342e8ee5d7a6e/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:18:19.530966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.531237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.531506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.553325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.555252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.561421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] MFA Rules not processed for user `e6cb7112373542648ed342e8ee5d7a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:19.562614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.640429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1127/2217] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:19.640975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af4a6-0d40-41dd-a4de-dd9e536fcfd1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.649842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.650509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.650781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.651052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] PATH_INFO: `/v3/users/3a6bae6e54b44a87b01b86e224a8f1e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.652210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6bae6e54b44a87b01b86e224a8f1e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.653145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.654172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.654628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.706287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6bae6e54b44a87b01b86e224a8f1e4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:19.713166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6bae6e54b44a87b01b86e224a8f1e4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:19.713903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6bae6e54b44a87b01b86e224a8f1e4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:19.724829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6bae6e54b44a87b01b86e224a8f1e4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:19.725311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf69de-b4c6-4d66-aeb4-773fabaff286 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.725632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9abbe02d-8717-4319-8fc8-5f038fa1a80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6bae6e54b44a87b01b86e224a8f1e4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:19.729035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.729648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.729930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.730323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.732020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.750098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1091/2218] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:19 2022] DELETE /identity/v3/users/3a6bae6e54b44a87b01b86e224a8f1e4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:18:19.755013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.755311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.755603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:19.778625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:19.779518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:19.780430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.798228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:19.799903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:19.808044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:19.809824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1128/2219] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:18:19.852101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1092/2220] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:19.863617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:19.865097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:19.933236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c907-c27d-4226-ae2e-562d35825150 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:19.995955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:19.996598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:19.996876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:19.997144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.009516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.010979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996347b7-533e-4636-bafc-98a094ca97f0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.022205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1129/2221] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:20.428542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.437057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:20.463024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df2fe4-c950-423d-b1e9-33fadf5a8f8f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.469834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.470541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.470877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.471150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] PATH_INFO: `/v3/projects/2d94d92a8124477d8358077e26c60cfe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.472362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d94d92a8124477d8358077e26c60cfe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.474583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.523169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d94d92a8124477d8358077e26c60cfe', 'request_id': 'req-140273d0-71be-4784-8025-fef10e4dd79b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:20.528608 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d94d92a8124477d8358077e26c60cfe', 'request_id': 'req-140273d0-71be-4784-8025-fef10e4dd79b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:20.534945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d94d92a8124477d8358077e26c60cfe', 'request_id': 'req-140273d0-71be-4784-8025-fef10e4dd79b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:20.549348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] The token cache is being invalidate because project 2d94d92a8124477d8358077e26c60cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:20.549604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:20.550212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-140273d0-71be-4784-8025-fef10e4dd79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:20.551790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1093/2222] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:20 2022] DELETE /identity/v3/projects/2d94d92a8124477d8358077e26c60cfe => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:20.556720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.557205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.557290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.577537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:20.579128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:20.639459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1130/2223] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:20.648801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.649090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.649363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.669785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:20.670856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:20.682490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1094/2224] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:20.686686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.696626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a62b7d-d513-4569-82a3-be3ce7da88ac None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.701364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.701966 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.702239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.702504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.703146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:18:20.703444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.705081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.717635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1131/2225] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:18:20.721874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.731207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a5be-f2fc-449b-9906-e703990bfb5b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.735913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.736554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.736841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.737188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.737832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.739730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.768917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1095/2226] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:18:20 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:18:20.773247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.784615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214471d9-ddd6-4217-8d00-fea78048a2c6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.816123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:20.845262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.845918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.846207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.846607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.847244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.849204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.874757 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1132/2227] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:18:20 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:18:20.876249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.887210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daef4a-9a32-46b7-adeb-304ff60925dd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.893472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.894165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.894437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.894751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.895448 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.897234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.907808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1096/2228] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:20.912141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.923120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b7b01-1267-41f4-97cc-2575051d9078 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.928798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.929400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.929675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.929938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.930579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.932310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9001118-4727-4331-bc6b-d51e426dab94 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.943416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1133/2229] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:18:20.947638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:20.958538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323829fa-4667-43d4-9faa-bf7f7dae5f78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:20.964337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:20.965043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:20.965323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:20.965598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] PATH_INFO: `/v3/projects/04c4f12270d54d5784bb9a12aced18c2/users/538b5483267549e6be369a653928b4b9/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:20.981288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c4f12270d54d5784bb9a12aced18c2, user_id=538b5483267549e6be369a653928b4b9, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:20.983957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:20.998554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1097/2230] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:18:20 2022] PUT /identity/v3/projects/04c4f12270d54d5784bb9a12aced18c2/users/538b5483267549e6be369a653928b4b9/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:21.004010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:21.004294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:21.004594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:21.023542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:21.026474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:21.108982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1134/2231] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:21.119129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:21.121211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:21.178407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:21.183137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5a8e1-164b-4f7d-abfa-199be2e671c6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:21.188535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:21.189224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:21.189818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:21.189818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:21.200090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:21.201800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed1ebe-d6a2-4d9d-afcf-a0e4a6dd68fb service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:21.220889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1098/2232] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:21.430304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:21.431710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:21.490626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493916bd-57ab-4f31-83df-4d333496a9c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:21.494908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:21.495527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:21.495839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:21.496392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:21.497969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:21.567827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:21.569266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:21.592769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1135/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:21.881869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:21.882392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:21.882512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:21.911170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] MFA Rules not processed for user `538b5483267549e6be369a653928b4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:21.912381 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:22.010378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1099/2234] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:22.062489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:22.063970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:22.076424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f5c79-7c95-4aba-bdd3-e1d68b6857e6 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:22.080991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:22.081742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:22.082311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:22.082533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:22.089980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:22.095078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:22.097584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5941f3b-0b52-4479-910c-b3626e4d8c26 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:22.108971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1136/2235] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:22.374079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:22.391781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7e82-84b9-4e2b-90df-84070f2f2ebb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:22.396302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:22.396971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:22.397343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:22.397695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:22.400122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:22.492636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:22.494167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:22.504388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1100/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:23.511189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b96e9-efd9-433f-842f-4879ea80cae2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:23.511716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b96e9-efd9-433f-842f-4879ea80cae2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:23.511716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b96e9-efd9-433f-842f-4879ea80cae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:23.531825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-191b96e9-efd9-433f-842f-4879ea80cae2 None None] MFA Rules not processed for user `538b5483267549e6be369a653928b4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:23.533069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-191b96e9-efd9-433f-842f-4879ea80cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:23.540712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1137/2237] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:24.235248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:24.236883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:24.255984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e571-5e04-4216-ab33-d96d6243c1ec service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:24.260718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:24.261716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:24.261716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:24.261878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] PATH_INFO: `/v3/users/5bddf7fd171842878e359beebbf57a67` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:24.263056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bddf7fd171842878e359beebbf57a67)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:24.264977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:24.315369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bddf7fd171842878e359beebbf57a67'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.323044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bddf7fd171842878e359beebbf57a67'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.323824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bddf7fd171842878e359beebbf57a67'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.333589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bddf7fd171842878e359beebbf57a67'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.334202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bddf7fd171842878e359beebbf57a67'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.340651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:24.356054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1101/2238] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:24 2022] DELETE /identity/v3/users/5bddf7fd171842878e359beebbf57a67 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:24.360807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:24.361088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:24.361354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:24.381115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:24.382355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:24.452908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1138/2239] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:24.463190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:24.464863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:24.530654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93e29e-2612-49db-bd5d-67cad7671c1a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:24.535553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:24.536788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:24.537129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:24.537391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:24.549016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:24.550911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:24.562595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1102/2240] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:18:24.852039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:24.854229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:24.870513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becacf5-d003-4cac-90ea-708c9c270cf2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:24.876565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:24.878791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:24.879402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:24.880742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] PATH_INFO: `/v3/projects/f66f956fcf914783be2e494b885f3348` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:24.883484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66f956fcf914783be2e494b885f3348)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:24.887502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:24.945699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66f956fcf914783be2e494b885f3348', 'request_id': 'req-7830c68d-006a-4209-8067-17188ca51440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.963518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66f956fcf914783be2e494b885f3348', 'request_id': 'req-7830c68d-006a-4209-8067-17188ca51440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.971110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66f956fcf914783be2e494b885f3348', 'request_id': 'req-7830c68d-006a-4209-8067-17188ca51440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.989057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] The token cache is being invalidate because project f66f956fcf914783be2e494b885f3348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:24.989354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.990196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7830c68d-006a-4209-8067-17188ca51440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:24.992180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1139/2241] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:24 2022] DELETE /identity/v3/projects/f66f956fcf914783be2e494b885f3348 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:26.488026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:26.489639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:26.517519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9c762-57ae-4a36-acd9-834349099c6f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:26.582175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:26.582823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:26.583100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:26.583374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] PATH_INFO: `/v3/users/91e0a548fbb24069afe1bae1107ff3a8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:26.584601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e0a548fbb24069afe1bae1107ff3a8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:26.586520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:26.642297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e0a548fbb24069afe1bae1107ff3a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:26.651155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e0a548fbb24069afe1bae1107ff3a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:26.651852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e0a548fbb24069afe1bae1107ff3a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:26.661010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e0a548fbb24069afe1bae1107ff3a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:26.661673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e0a548fbb24069afe1bae1107ff3a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:26.680903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1103/2242] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:26 2022] DELETE /identity/v3/users/91e0a548fbb24069afe1bae1107ff3a8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:26.685718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:26.686621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:26.687080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:26.709360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:26.710592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:26.768037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1140/2243] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:26.778107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:26.779532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:26.791670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:26.839603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac94be-bc22-4f0b-bb90-57c9b4ab5603 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:26.843565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:26.844160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:26.844434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:26.844733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:26.854137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:26.855577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:26.864962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1104/2244] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:18:27.145419 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:27.146899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:27.159571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b01566-aeda-4824-81c0-6dff80c698bf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:27.165537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:27.166174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:27.166448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:27.166713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] PATH_INFO: `/v3/projects/6b7d4dd115f647ba8856c2420be2e7e1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:27.167946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7d4dd115f647ba8856c2420be2e7e1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:27.169989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:27.177006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:27.220378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7d4dd115f647ba8856c2420be2e7e1', 'request_id': 'req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.229421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7d4dd115f647ba8856c2420be2e7e1', 'request_id': 'req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.235968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7d4dd115f647ba8856c2420be2e7e1', 'request_id': 'req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.246443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] The token cache is being invalidate because project 6b7d4dd115f647ba8856c2420be2e7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:27.246639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.247472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.249031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1141/2245] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:27 2022] DELETE /identity/v3/projects/6b7d4dd115f647ba8856c2420be2e7e1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:27.253159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:27.263278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d0f4a-2e3d-4805-bd9e-a14f5ab9f11b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:27.326859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:27.327495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:27.327775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:27.328043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] PATH_INFO: `/v3/users/99e28ada8e9f4b90bbb41a0d00e6a505` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:27.329228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e28ada8e9f4b90bbb41a0d00e6a505)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:27.331164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:27.381814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e28ada8e9f4b90bbb41a0d00e6a505'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.389090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e28ada8e9f4b90bbb41a0d00e6a505'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.389836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e28ada8e9f4b90bbb41a0d00e6a505'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.399042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e28ada8e9f4b90bbb41a0d00e6a505'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.399704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e28ada8e9f4b90bbb41a0d00e6a505'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.417265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1105/2246] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:27 2022] DELETE /identity/v3/users/99e28ada8e9f4b90bbb41a0d00e6a505 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:18:27.842726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:27.844452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:27.858077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d0ac-5919-42f1-ac61-ba5cfbc2042f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:27.917119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:27.917758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:27.918035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:27.918303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] PATH_INFO: `/v3/projects/fbceda11ed964612a4a6945c32bde428` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:27.919465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbceda11ed964612a4a6945c32bde428)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:27.921447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:27.972137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbceda11ed964612a4a6945c32bde428', 'request_id': 'req-05001ff9-4907-48cf-bbde-27036510fdd2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.984586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbceda11ed964612a4a6945c32bde428', 'request_id': 'req-05001ff9-4907-48cf-bbde-27036510fdd2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:27.994596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbceda11ed964612a4a6945c32bde428', 'request_id': 'req-05001ff9-4907-48cf-bbde-27036510fdd2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.007788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] The token cache is being invalidate because project fbceda11ed964612a4a6945c32bde428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:28.008248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.009247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.011166 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1142/2247] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:27 2022] DELETE /identity/v3/projects/fbceda11ed964612a4a6945c32bde428 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:28.021922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:28.023553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:28.037404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc671-4557-47c0-a54d-9487cf363997 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:28.069954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:28.103602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:28.104343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:28.104652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:28.104926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] PATH_INFO: `/v3/users/e6cb7112373542648ed342e8ee5d7a6e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:28.106076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6cb7112373542648ed342e8ee5d7a6e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:28.108000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:28.172268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6cb7112373542648ed342e8ee5d7a6e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.179923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6cb7112373542648ed342e8ee5d7a6e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.180818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6cb7112373542648ed342e8ee5d7a6e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.189712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6cb7112373542648ed342e8ee5d7a6e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.190380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1e452e6d-3341-4591-b75e-aa6ad0e18ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6cb7112373542648ed342e8ee5d7a6e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.209083 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1106/2248] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:18:28 2022] DELETE /identity/v3/users/e6cb7112373542648ed342e8ee5d7a6e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:28.482954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:18:28.484749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:28.505693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001ff9-4907-48cf-bbde-27036510fdd2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:18:28.527884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:28.567157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:18:28.567801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:28.568082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:28.568362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] PATH_INFO: `/v3/projects/127fe9a5e23a4ed385b5ce9771760569` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:28.569526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127fe9a5e23a4ed385b5ce9771760569)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:18:28.571490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:18:28.622482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127fe9a5e23a4ed385b5ce9771760569', 'request_id': 'req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.630475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127fe9a5e23a4ed385b5ce9771760569', 'request_id': 'req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.639070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127fe9a5e23a4ed385b5ce9771760569', 'request_id': 'req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.649791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] The token cache is being invalidate because project 127fe9a5e23a4ed385b5ce9771760569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:18:28.650078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.650729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4b73a81b-9f7a-4ec0-a09a-472d393c9d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:18:28.652245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1143/2249] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:18:28 2022] DELETE /identity/v3/projects/127fe9a5e23a4ed385b5ce9771760569 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:18:38.209861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:38.646915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:18:40.142362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:18:40.142839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:18:40.142935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:18:40.162461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] MFA Rules not processed for user `edbf26fde0254259a540f31139fc1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:18:40.163881 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:18:40.222415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1107/2250] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:18:50.215371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:19:16.157537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a3a4-eb7a-4c18-a675-2a604730d434 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:19:16.158094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a3a4-eb7a-4c18-a675-2a604730d434 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:19:16.158094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a3a4-eb7a-4c18-a675-2a604730d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:19:16.180253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bd77a3a4-eb7a-4c18-a675-2a604730d434 None None] MFA Rules not processed for user `ed4711b7691643109007e173fee1c405`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:19:16.181499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd77a3a4-eb7a-4c18-a675-2a604730d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:19:16.241930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1144/2251] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:19:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:19:17.380751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:19:17.382434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:19:17.460705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50950368-ccdc-4c4e-8a9c-c2549fb28f5f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:19:17.466212 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:19:17.466844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:19:17.467126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:19:17.467395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:19:17.469214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:19:17.480601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:19:17.482201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:19:17.491853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1108/2252] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:19:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:19:26.234499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:19:27.488271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=32 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:19:45.969796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddf252-630c-4b45-b6d5-ecb66bdd5fc5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:19:45.970337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddf252-630c-4b45-b6d5-ecb66bdd5fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:19:45.970526 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddf252-630c-4b45-b6d5-ecb66bdd5fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:19:45.990084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-40ddf252-630c-4b45-b6d5-ecb66bdd5fc5 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:19:45.991572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-40ddf252-630c-4b45-b6d5-ecb66bdd5fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:19:46.056892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1145/2253] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:19:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:19:46.099962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:19:46.101525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:19:46.172939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc9167-1cf0-4a93-ae55-c96091eb669e service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:19:46.179557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:19:46.180179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:19:46.180460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:19:46.180765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:19:46.191459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:19:46.193511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e041dd-ac58-47a3-8b2c-dc09779d8f92 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:19:46.205080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1109/2254] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:19:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:19:56.047637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:19:56.205491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=32 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:20:04.074834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d22-426a-4389-84a4-91a230667353 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:04.075346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d22-426a-4389-84a4-91a230667353 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:04.075346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d22-426a-4389-84a4-91a230667353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:04.095230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ea759d22-426a-4389-84a4-91a230667353 None None] MFA Rules not processed for user `538b5483267549e6be369a653928b4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:04.096484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ea759d22-426a-4389-84a4-91a230667353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:04.151829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1146/2255] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:20:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:20:05.672244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:05.672858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:05.672858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:05.674359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1110/2256] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:20:05.682861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ea9-0cb4-4ad7-8a1a-9fcabb6ac4f5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:05.683350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ea9-0cb4-4ad7-8a1a-9fcabb6ac4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:05.683470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ea9-0cb4-4ad7-8a1a-9fcabb6ac4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:05.703889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-67225ea9-0cb4-4ad7-8a1a-9fcabb6ac4f5 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:05.705222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-67225ea9-0cb4-4ad7-8a1a-9fcabb6ac4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:05.783128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1147/2257] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:20:05.795720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:05.797726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:05.873091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9a8f-4782-4a61-b6ff-85e89b4eac58 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:05.876975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:05.877766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:05.878744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:05.879333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:05.892446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:05.894026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d99e8-ea39-4e6c-a16a-b40d1b99f826 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:05.906428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1111/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:20:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:20:05.987132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:05.987725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:05.988285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:05.990030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1148/2259] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:20:05.995827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b477-ee69-4b95-81ce-8f41680c07a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:05.996090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b477-ee69-4b95-81ce-8f41680c07a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:05.997226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b477-ee69-4b95-81ce-8f41680c07a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:06.021386 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a574b477-ee69-4b95-81ce-8f41680c07a8 None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:06.022625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a574b477-ee69-4b95-81ce-8f41680c07a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:06.085389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1112/2260] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:20:06.097212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:06.098923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:06.113429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf6991-7245-44ea-b264-0579652c1a37 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:06.117277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:06.117956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:06.118240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:06.118511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:06.126827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:06.128699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c654bd7-01ea-475b-919d-8e5d5551a83c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:06.137207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1149/2261] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:20:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:20:06.483788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f80b-f75b-452e-9045-493caf4fc2af None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:06.484078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f80b-f75b-452e-9045-493caf4fc2af None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:06.484371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f80b-f75b-452e-9045-493caf4fc2af None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:06.485767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1113/2262] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:20:06.491441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f023a-8295-4781-a959-88772eea9546 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:06.491734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f023a-8295-4781-a959-88772eea9546 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:06.492003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f023a-8295-4781-a959-88772eea9546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:06.513071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e51f023a-8295-4781-a959-88772eea9546 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:06.514304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e51f023a-8295-4781-a959-88772eea9546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:06.523789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1150/2263] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:20:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:20:06.534313 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f80b-f75b-452e-9045-493caf4fc2af None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:06.547071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f80b-f75b-452e-9045-493caf4fc2af None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:06.550927 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:06.551613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:06.551898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:06.552170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:06.563389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:06.564871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:06.575529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1114/2264] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:20:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:20:13.131521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:13.132038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:13.132144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:13.151982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:13.153312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:13.160494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1151/2265] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:20:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:20:13.178797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:13.193253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03317fac-5a28-4660-98bc-2cb4afc6e2e0 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:13.208127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:13.208889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:13.209284 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:13.209708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:13.219711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:13.221922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:13.231268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1115/2266] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:20:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:20:23.146712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=68 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:20:23.227999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=61 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:20:44.410464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:44.420633 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a66f-0692-43f7-b1ba-1db8ce09d2b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:44.424522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:44.425208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:44.425550 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:44.425876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:44.428089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:44.444110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:44.445703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d03e4-4ec1-4370-b095-d9185b1dc786 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:44.457171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1152/2267] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:20:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:20:45.412586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:45.422913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fb7ec-dd78-4b6f-9b05-f6f7b82e969d service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:45.427216 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:45.427815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:45.428093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:45.428360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:45.430246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:45.456344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:45.457875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:45.469689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1116/2268] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:20:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:20:48.947413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:48.947904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:48.948048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:48.968823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:20:48.969997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:20:48.977813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1153/2269] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:20:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:20:49.001005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:20:49.012200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b04d3-cf97-462f-b40d-c414a1978814 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:20:49.017547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:20:49.018153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:20:49.018429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:20:49.018889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:20:49.027859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:20:49.029483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:20:49.037954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1117/2270] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:20:58.963930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:20:59.034875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=12 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:28.032653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:28.035494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:28.059370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81042a-0926-4c9a-bbb1-a5da4a89cd1a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:28.066577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:28.067185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:28.067462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:28.067726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:28.069895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:28.080905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:28.082540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:28.091798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1154/2271] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:21:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:21:38.087636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:38.470555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:38.472105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:38.486011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef56de-8f18-459a-bc8c-0c5442a531bb service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:38.490550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:38.491184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:38.491504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:38.491765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:38.493767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:38.504458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:38.505958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:38.514265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1118/2272] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:21:41.658010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:41.659570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:41.726972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bcb16-d5fb-40cc-8088-c1c96fc8482d service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:41.731964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:41.732647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:41.732923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:41.733187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] PATH_INFO: `/v3/users/538b5483267549e6be369a653928b4b9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:41.734321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538b5483267549e6be369a653928b4b9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:41.736234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:41.792842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538b5483267549e6be369a653928b4b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:41.799916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538b5483267549e6be369a653928b4b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:41.812648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538b5483267549e6be369a653928b4b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:41.821679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538b5483267549e6be369a653928b4b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:41.822417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d201520d-caa3-4b1c-aca5-35004b969945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538b5483267549e6be369a653928b4b9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:41.828553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1155/2273] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:21:41 2022] DELETE /identity/v3/users/538b5483267549e6be369a653928b4b9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:21:42.031652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.044259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f71165-48cf-450e-ad45-fa064a77567e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.106111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.106752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.107034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.107309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] PATH_INFO: `/v3/projects/04c4f12270d54d5784bb9a12aced18c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.108534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c4f12270d54d5784bb9a12aced18c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.110597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.163573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c4f12270d54d5784bb9a12aced18c2', 'request_id': 'req-bf1d72aa-edc1-4661-80c4-61c23227572b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:42.170352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c4f12270d54d5784bb9a12aced18c2', 'request_id': 'req-bf1d72aa-edc1-4661-80c4-61c23227572b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:42.176936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c4f12270d54d5784bb9a12aced18c2', 'request_id': 'req-bf1d72aa-edc1-4661-80c4-61c23227572b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:42.191493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] The token cache is being invalidate because project 04c4f12270d54d5784bb9a12aced18c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:21:42.191784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:42.192443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bf1d72aa-edc1-4661-80c4-61c23227572b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:21:42.194046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1119/2274] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:21:42 2022] DELETE /identity/v3/projects/04c4f12270d54d5784bb9a12aced18c2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:21:42.199282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.199476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.199747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.219508 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:21:42.220733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:42.284383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1156/2275] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:21:42.292915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.293203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.293478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.313893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:21:42.315109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:42.323916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1120/2276] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:21:42.328301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.338406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15c4ff-20b4-4825-8e8b-41351f2882c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.344075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.344704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.345076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.345339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.345975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:21:42.346274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.347907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.359882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1157/2277] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:21:42.364244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.373495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8351cc-c7e5-42f2-bd63-dffe6dee1575 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.378376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.378969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.379248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.379519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.380165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.382106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.413752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1121/2278] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:21:42 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:21:42.418543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.428667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06315-266a-4ea9-8d15-3069a76f5b39 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.432450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:42.487925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.488551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.488831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.489095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.489721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.491675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.514035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1158/2279] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:21:42 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:21:42.518364 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.529379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901769-3c9c-4497-a8af-671154fecff2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.535312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.535912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.536187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.536440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.537125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.538728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.549822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1122/2280] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:21:42.553774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.564193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14c090-7371-40dc-abb9-599de86c5f0b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.569149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.569742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.570023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.570293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.570935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.572574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16818f6-846d-4841-8b5f-ab9f99a51713 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.584288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1159/2281] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:21:42.588365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.599018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a769b05-74d3-4192-b177-cb7492827559 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.604127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.604825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.605077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.605345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] PATH_INFO: `/v3/projects/95d488336b70426a8fec2c87d4741bd2/users/914eb02a57324788940458be01b484ba/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.621145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d488336b70426a8fec2c87d4741bd2, user_id=914eb02a57324788940458be01b484ba, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.623801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.640045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1123/2282] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:21:42 2022] PUT /identity/v3/projects/95d488336b70426a8fec2c87d4741bd2/users/914eb02a57324788940458be01b484ba/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:21:42.650434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.650752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.651046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.679939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] MFA Rules not processed for user `914eb02a57324788940458be01b484ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:21:42.681188 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:42.749466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1160/2283] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:21:42.762671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:42.764833 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:42.781028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:42.825950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb89988-b7fe-4086-8a16-319539cfee68 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:42.829982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:42.830585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:42.830860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:42.831137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:42.840837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:42.842286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:42.851347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1124/2284] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:21:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:21:52.745610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:52.847639 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=20 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:21:59.777651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:21:59.779273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:59.871107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24ea99-88f4-45a9-be12-634fcda2bf3f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:21:59.878694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:21:59.879320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:21:59.879618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:21:59.880010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:21:59.881656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:21:59.940185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:21:59.941887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:21:59.951146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1161/2285] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:04.801757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:04.803341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:04.820824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d55e7-4648-4bb4-915e-b9443f2872c7 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:04.826645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:04.827406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:04.827687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:04.828039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:04.829971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:04.843626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:04.845366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bedc7-143f-4b5a-b890-b6daf94a7665 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:04.856570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1125/2286] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:05.486680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:05.554790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478dadc-9b7e-47a5-8a09-dcc4688073b2 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:05.559681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:05.560418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:05.560847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:05.561207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] PATH_INFO: `/v3/users/914eb02a57324788940458be01b484ba` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:05.562445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914eb02a57324788940458be01b484ba)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:05.564661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:05.628459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:05.637817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914eb02a57324788940458be01b484ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:05.651549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914eb02a57324788940458be01b484ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:05.670753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914eb02a57324788940458be01b484ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:05.682288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914eb02a57324788940458be01b484ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:05.683175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3952136d-857b-4ce4-ad38-ff7d25da1fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914eb02a57324788940458be01b484ba'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:05.690257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1162/2287] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:22:05 2022] DELETE /identity/v3/users/914eb02a57324788940458be01b484ba => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:22:05.695204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:05.695485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:05.695750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:05.724941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:05.726532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:05.857323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:05.857754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:05.858124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:05.880748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] MFA Rules not processed for user `edbf26fde0254259a540f31139fc1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:05.882052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:05.939229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1126/2288] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:05.949672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:05.951518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:05.972025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1163/2289] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:22:06.012327 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dfd43-28df-46c4-96d5-8fe221ee8922 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:06.015962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:06.016614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.016948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.017172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.026657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:06.028170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb210dd-fffe-4771-882b-2d45157d7bc2 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:06.038304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1127/2290] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:06.534594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:06.536218 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:06.553083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e976913-4c40-4d15-8e41-d8c908581999 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:06.557874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:06.558691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.559127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.559545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] PATH_INFO: `/v3/projects/95d488336b70426a8fec2c87d4741bd2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.560880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d488336b70426a8fec2c87d4741bd2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:06.563632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:06.632921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d488336b70426a8fec2c87d4741bd2', 'request_id': 'req-54497666-bb4c-4cdd-916c-96d8e480d017'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:06.642252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d488336b70426a8fec2c87d4741bd2', 'request_id': 'req-54497666-bb4c-4cdd-916c-96d8e480d017'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:06.651135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d488336b70426a8fec2c87d4741bd2', 'request_id': 'req-54497666-bb4c-4cdd-916c-96d8e480d017'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:06.665405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] The token cache is being invalidate because project 95d488336b70426a8fec2c87d4741bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:22:06.665692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:06.666696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-54497666-bb4c-4cdd-916c-96d8e480d017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:22:06.668536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1164/2291] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:22:06 2022] DELETE /identity/v3/projects/95d488336b70426a8fec2c87d4741bd2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:22:06.674769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.675458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.675734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.697531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:06.699445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:06.770121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1128/2292] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:22:06.774852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.775640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.775640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.798078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:06.799286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:06.807157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1165/2293] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:06.811804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:06.822171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46be965-5459-4a5b-9c55-ba8096084009 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:06.826742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:06.827352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.827628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.827895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.828577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:22:06.828876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:06.830513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:06.843363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1129/2294] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:22:06.847724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:06.858253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabcf5e-1460-42f9-94e8-a531ffa4b5fc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:06.863583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:06.864197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.864870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.864870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.865413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:06.867347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:06.895809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1166/2295] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:22:06 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:22:06.899942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:06.910672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee366-6cff-4171-b633-fc8e9f7bb711 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:06.925539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:06.975955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:06.977306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:06.977584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:06.977854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:06.978532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:06.980451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:07.008125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1130/2296] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:22:06 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:22:07.010215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:07.024393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c7c03-8b7c-45fd-87dd-841f0c5155bd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:07.032535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:07.033223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:07.033515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:07.033783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:07.034440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:07.036094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:07.047395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1167/2297] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:22:07.051661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:07.062454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b44c4-ea57-4938-b5a0-a0b8a14d04ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:07.067288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:07.068008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:07.068380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:07.068807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:07.069850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:07.071502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c0b0-d014-4850-b8ac-d9a795adcb84 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:07.085132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1131/2298] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:22:07.086881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:07.094130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:07.098172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f2830-96ee-4f4e-8b4b-4349aecaed42 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:07.106565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:07.107651 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:07.108017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:07.108368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] PATH_INFO: `/v3/projects/af70fccdef544a499725509ccd8c545f/users/3deb7adb33004f1097574470754611ec/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:07.129416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af70fccdef544a499725509ccd8c545f, user_id=3deb7adb33004f1097574470754611ec, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:07.132187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:07.154201 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1168/2299] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:22:07 2022] PUT /identity/v3/projects/af70fccdef544a499725509ccd8c545f/users/3deb7adb33004f1097574470754611ec/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:22:07.159002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8b6c-29dd-488b-a672-996482ac0b15 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:07.159312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8b6c-29dd-488b-a672-996482ac0b15 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:07.159585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8b6c-29dd-488b-a672-996482ac0b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:07.178648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-111b8b6c-29dd-488b-a672-996482ac0b15 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:07.179768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-111b8b6c-29dd-488b-a672-996482ac0b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:07.244278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1132/2300] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:07.250978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:07.252778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:07.320067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5c826-c228-4fa0-9885-525f5d158414 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:07.321589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:07.322246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:07.322548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:07.322908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:07.333499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:07.334825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:07.344516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1169/2301] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:11.686856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:11.687701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:11.688122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:11.743826 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] MFA Rules not processed for user `3deb7adb33004f1097574470754611ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:11.745263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:11.841646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1133/2302] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:11.845655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:11.859554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4698f88-9ae0-48e8-8063-ad29ce0fd1ff service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:11.866370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:11.867378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:11.867775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:11.868131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:11.869347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:11.871095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:11.919829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1170/2303] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:22:11 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:22:11.922772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:11.937928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa95ba4-a6d7-485e-bf20-45d3a86d1e1b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:11.977117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:12.017579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:12.018402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:12.018860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:12.019363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:12.020058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:12.022213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:12.046875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1134/2304] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:22:11 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:22:12.051060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:12.061984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9a91f-2283-4c62-9048-8805877f2303 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:12.067174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:12.067805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:12.068087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:12.068361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:12.069067 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:12.071399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:12.082802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1171/2305] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:22:12.087211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:12.098450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146a0f-1492-4ee7-8b20-242d09d9aa57 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:12.104990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:12.105684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:12.106106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:12.106403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] PATH_INFO: `/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c/users/9ef733aae36e477b835993ef48e7ca1f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:12.123928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f46e3e7c6a41fda0f140b80e320a0c, user_id=9ef733aae36e477b835993ef48e7ca1f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:12.126790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:12.143742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1135/2306] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:22:12 2022] PUT /identity/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c/users/9ef733aae36e477b835993ef48e7ca1f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:22:12.148024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:12.163768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671db34-98b2-4676-9756-ee272751b352 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:12.228771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:12.229450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:12.229669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:12.229982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:12.230701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:12.232668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:12.256529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1172/2307] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:22:12.260673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:12.275662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9e269-7f4e-43e4-a5fb-87cd476630a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:12.280659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:12.281332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:12.281611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:12.281875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] PATH_INFO: `/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c/users/9ef733aae36e477b835993ef48e7ca1f/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:12.283841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f46e3e7c6a41fda0f140b80e320a0c, user_id=9ef733aae36e477b835993ef48e7ca1f, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:12.286626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce1b26-0174-426c-bb3b-a5c1b7d16cfe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:12.305669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1136/2308] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:22:12 2022] PUT /identity/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c/users/9ef733aae36e477b835993ef48e7ca1f/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:22:12.985516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:12.989129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:13.070004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:13.110035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9949011-3874-4f59-8725-25f566e89ec4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:13.196460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:13.197186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:13.197469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:13.197743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:13.199600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:13.214002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:13.215813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:13.228396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1173/2309] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:18.982834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:18.983441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:18.983441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:19.019096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] MFA Rules not processed for user `9ef733aae36e477b835993ef48e7ca1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:19.020488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:19.128766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1137/2310] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:19.142388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:19.155079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51822ff-0085-4142-b5e8-c91d63c04aa2 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:19.161128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:19.161790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:19.162068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:19.162338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:19.221234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:19.222907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe92740-02e5-462d-9a76-cfc53db48549 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:19.232565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1174/2311] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:20.605861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:20.619767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527e709-a676-44d0-9cf4-b5bfce607de7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:20.628739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:20.630059 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:20.630510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:20.631035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:20.633309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:20.717947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:20.721438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:20.733413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1138/2312] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:21.523110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:21.523635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:21.523635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:21.555002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] MFA Rules not processed for user `ed4711b7691643109007e173fee1c405`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:21.556363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:21.616123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1175/2313] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:21.976537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:21.980945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:22.043264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd0000-e372-4c25-a86a-3f6229ea8719 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:22.046920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:22.047535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:22.047814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:22.048088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:22.049892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:22.060418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:22.061975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf75eb9-c0bd-4667-b7d2-1fc5230a1655 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:22.071755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1139/2314] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:22.454745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:22.456194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:22.474056 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:22.575339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21594b17-21ef-4277-acc5-8eab9d7d659d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:22.591420 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:22.591574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:22.592038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:22.593009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:22.595035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:22.612427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:22.615588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:22.628997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1176/2315] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:30.377669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:30.378294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:30.378294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:30.399226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] MFA Rules not processed for user `5bb45c233ca243388bee05dede63415f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:30.400449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:30.407346 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1140/2316] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:32.624047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:33.729315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:33.747212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2daab-9d70-4fcc-92bb-c8533096f6d9 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:33.753843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:33.755128 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:33.755963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:33.756457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:33.758770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:33.776879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:33.778561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95839079-8d86-4fd3-8700-e3ca08660d97 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:33.794335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1177/2317] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:39.741070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:39.755075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79955-c321-4569-85cf-25979b1dd20d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:39.761209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:39.761982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:39.762275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:39.762704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:39.764621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:39.843837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:39.845610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:39.855960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1141/2318] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:43.272085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:43.272545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:43.272675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:43.291595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] MFA Rules not processed for user `3deb7adb33004f1097574470754611ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:22:43.293171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:43.300826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1178/2319] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:22:43.692900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:22:43.702238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8129f95-b4ca-431e-aa26-4694c6226eda service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:22:43.705941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:22:43.706538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:22:43.706815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:22:43.707142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:22:43.709029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:22:43.718568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:22:43.719992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:22:43.727965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1142/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:22:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:22:53.286824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:22:53.724619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=62 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:02.446728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:02.448940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:02.466903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253843a-c9cd-49fc-8f87-5d202bbfab71 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:02.471445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:02.472074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:02.472351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:02.472673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:02.482246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:02.483669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074bc5-b62a-4b11-b8a0-75c279d924a8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:02.493579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1179/2321] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:12.489379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:36.202082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:36.210886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10901627-f72f-46ed-86c9-dac7e912f9fc service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:36.214797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:36.215390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:36.215669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:36.215944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:36.217829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:36.285157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:36.286600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6461b7a-7f80-4a37-a09f-df82e89a8fc8 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:36.297122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1143/2322] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:39.434021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.434589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.436448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.438753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1180/2323] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:23:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:23:39.446912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.447759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.448295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.476801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] MFA Rules not processed for user `ca82b32794ef4e37b9c2608a6e78930f`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:39.478058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:39.487698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1144/2324] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:23:39.501345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:39.503695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:39.520208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:39.521651 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:39.536278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5783cb-165f-40a0-903f-676a208e4838 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:39.537982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52602c0-e2b6-4706-b9af-58d71013d522 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:39.540754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:39.541392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.541673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.541945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] PATH_INFO: `/v3/users/edbf26fde0254259a540f31139fc1a51` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.543472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbf26fde0254259a540f31139fc1a51)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:39.544987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:39.545588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:39.545704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.546131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.546466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.559590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:39.561844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:39.582912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1181/2325] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:39.611025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbf26fde0254259a540f31139fc1a51'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.618329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbf26fde0254259a540f31139fc1a51'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.619020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbf26fde0254259a540f31139fc1a51'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.626728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbf26fde0254259a540f31139fc1a51'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.627617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-288952d0-b1a9-4e59-b0b9-80968e302f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbf26fde0254259a540f31139fc1a51'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.644815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1145/2326] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:23:39 2022] DELETE /identity/v3/users/edbf26fde0254259a540f31139fc1a51 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:39.816907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:39.828161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66f31-7d23-4194-b8ea-35cc19456bb5 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:39.885716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:39.886708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.887301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.887573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] PATH_INFO: `/v3/projects/76d489fb3d144cc3abf03a842e70636d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.888784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d489fb3d144cc3abf03a842e70636d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:39.890691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:39.939418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d489fb3d144cc3abf03a842e70636d', 'request_id': 'req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.949189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d489fb3d144cc3abf03a842e70636d', 'request_id': 'req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.955615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d489fb3d144cc3abf03a842e70636d', 'request_id': 'req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.966937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] The token cache is being invalidate because project 76d489fb3d144cc3abf03a842e70636d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:23:39.967261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.968016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6c947fa1-d2fb-423d-8dd0-35fed7f3952d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:39.969638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1182/2327] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:23:39 2022] DELETE /identity/v3/projects/76d489fb3d144cc3abf03a842e70636d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:39.974890 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:39.975176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:39.975453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:39.996161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:39.997367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:40.048369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:40.058822 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1146/2328] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:40.066478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.066764 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.067037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.087318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:40.088601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:40.097623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1183/2329] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:40.102487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.112592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b690d39-45fc-41a1-89df-a5e0592d6ce5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.117843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.118567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.118861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.119225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.119918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:23:40.120246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.122118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.133439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1147/2330] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:23:40.137813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.147529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653d968-97b7-441a-95dd-945702b80e98 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.152907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.153504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.153781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.154047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.154686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.156902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.189657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1184/2331] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:23:40 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:40.193945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.204130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae2721-0e6b-47ae-8219-3c62171f3a95 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.262468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.262990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.263270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.263551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.264337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.266306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.292821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1148/2332] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:23:40 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:40.297108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.309159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2882113-74cd-488f-85fd-603a80a9752f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.314923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.315511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.315780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.316043 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.316719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.318326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.329030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1185/2333] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:40.333054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.343538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a167-9f8d-4780-beb7-f558ee40695a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.349516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.350101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.350376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.350640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.351294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.352949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f42f3-795f-4237-8ff1-a1d0ff8e0fc1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.363199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1149/2334] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:40.367395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.377615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0a4e-faa5-48b6-b6ce-16989013659e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.382395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.383023 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.383298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.383561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] PATH_INFO: `/v3/projects/4558b7fc3e804764b3f3e72687fb7e76/users/4b9abe14122b40a9bacab814d8b8ba11/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.398250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4558b7fc3e804764b3f3e72687fb7e76, user_id=4b9abe14122b40a9bacab814d8b8ba11, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.400907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.416680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1186/2335] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:40 2022] PUT /identity/v3/projects/4558b7fc3e804764b3f3e72687fb7e76/users/4b9abe14122b40a9bacab814d8b8ba11/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:23:40.429316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.429598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.429953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.460337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] MFA Rules not processed for user `4b9abe14122b40a9bacab814d8b8ba11`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:40.461584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:40.534610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1150/2336] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:40.546358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:40.548753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:40.563845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:40.625830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bd97b-8d6b-4bf3-bd0c-e263c8198677 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:40.631498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:40.632097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:40.632367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:40.632672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:40.640303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:40.641914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec134e-6212-4650-8cf4-c9e3f168f614 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:40.650488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1187/2337] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:43.382186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:43.439854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:43.446238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae129747-5b67-4cfa-b622-0f62428361a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:43.451797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:43.452429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:43.452736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:43.453004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] PATH_INFO: `/v3/users/4b9abe14122b40a9bacab814d8b8ba11` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:43.454256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9abe14122b40a9bacab814d8b8ba11)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:43.456243 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:43.504067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9abe14122b40a9bacab814d8b8ba11'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:43.510944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9abe14122b40a9bacab814d8b8ba11'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:43.511591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9abe14122b40a9bacab814d8b8ba11'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:43.520043 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9abe14122b40a9bacab814d8b8ba11'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:43.520747 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9abe14122b40a9bacab814d8b8ba11'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:43.537834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1151/2338] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:23:43 2022] DELETE /identity/v3/users/4b9abe14122b40a9bacab814d8b8ba11 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:43.543010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:43.543778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:43.543778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:43.563437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:43.564712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:43.626968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1188/2339] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:43.637098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:43.638831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:43.699230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ce6bb-2a5a-4489-935c-124f85ea4124 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:43.703763 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:43.704451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:43.704759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:43.705029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:43.721903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:43.723819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7958d-1e7b-4443-b473-ca93af72f476 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:43.737706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1152/2340] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:44.013715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.015976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:44.034930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4829c-45d9-4c69-92b2-7207b0a46ac7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.041044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.041576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.041967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.042058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] PATH_INFO: `/v3/projects/4558b7fc3e804764b3f3e72687fb7e76` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.043831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4558b7fc3e804764b3f3e72687fb7e76)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.045784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.095605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4558b7fc3e804764b3f3e72687fb7e76', 'request_id': 'req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:44.105786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4558b7fc3e804764b3f3e72687fb7e76', 'request_id': 'req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:44.112872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4558b7fc3e804764b3f3e72687fb7e76', 'request_id': 'req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:44.125793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] The token cache is being invalidate because project 4558b7fc3e804764b3f3e72687fb7e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:23:44.126079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:44.126901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b8ea07bc-0bf9-4a26-9adc-88684aa36f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:44.128597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1189/2341] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:23:44 2022] DELETE /identity/v3/projects/4558b7fc3e804764b3f3e72687fb7e76 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:44.133598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.133885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.134155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.154794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:44.156017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:44.218877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1153/2342] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:44.226852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.227133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.227397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.247685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:44.248910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:44.258561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1190/2343] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:44.262775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.273146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abd5e-edce-4c34-a807-faa34c4c2353 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.278262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.278855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.279133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.279403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.280086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:23:44.280377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.282038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.294178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1154/2344] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:23:44.298438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.307990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4fe80-e9a3-43fc-ab63-14c6677c291c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.313594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.314310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.314586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.314851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.315475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.317442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.332467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:44.352120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1191/2345] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:23:44 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:44.353428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.360124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:44.364362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a804-f5f9-4c64-8f57-fc80dd43ffd2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.423085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.423723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.423972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.424227 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.424873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.426807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.450384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1155/2346] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:23:44 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:44.454606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.465100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80711a2d-4a40-4e05-9f5a-ee806d66eb98 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.469869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.470485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.470769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.471018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.471672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.473331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.483566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1192/2347] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:44.487696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.497841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa7a6-ca2b-42a5-8a45-dbc9d2ccb919 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.503007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.503603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.503886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.504157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.504841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.506434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035de8a-f6bb-4af7-aff6-9d41ab7d2c29 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.517768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1156/2348] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:23:44.522045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.532482 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122f64-7a20-45cc-add4-8e46af163e90 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.538519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.539169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.539445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.539710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] PATH_INFO: `/v3/projects/f7b8f402cb2848219899c8f025ec39dd/users/2404b512ac7b4f69ad39d4e9c06e9ff2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.555903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b8f402cb2848219899c8f025ec39dd, user_id=2404b512ac7b4f69ad39d4e9c06e9ff2, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.558616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.575135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1193/2349] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:44 2022] PUT /identity/v3/projects/f7b8f402cb2848219899c8f025ec39dd/users/2404b512ac7b4f69ad39d4e9c06e9ff2/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:44.580324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e569b-309f-433e-8989-cfdba56b5060 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.580702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e569b-309f-433e-8989-cfdba56b5060 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.580976 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e569b-309f-433e-8989-cfdba56b5060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.601680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-867e569b-309f-433e-8989-cfdba56b5060 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:44.602905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-867e569b-309f-433e-8989-cfdba56b5060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:44.662368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1157/2350] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:23:44.672573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:44.674434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:44.731617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe0f7f-ebd6-406a-b62b-04c289f2fc73 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:44.736003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:44.736624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:44.736895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:44.737158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:44.746720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:44.748147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:44.757620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1194/2351] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:45.436310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.436870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.436870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.472441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] MFA Rules not processed for user `2404b512ac7b4f69ad39d4e9c06e9ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:45.473683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:45.538324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1158/2352] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:45.543147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.552740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625065e-899d-4a42-bed4-b6600f3b19dc service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.557967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.558628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.558911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.559180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.559810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.562032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.603722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1195/2353] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:23:45 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:45.604926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.615882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37113983-3788-4501-9bf7-3924108e3e31 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.624637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:45.673863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.674496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.674773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.675050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.675720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.677754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.700393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1159/2354] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:23:45 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:23:45.704665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.715523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36514d4f-21da-4428-8838-513f475ddf28 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.721148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.721750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.722054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.722328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.722975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.724710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.735396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1196/2355] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:45.739675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.750269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597e32b-894d-45cd-ab5e-dfdc0753cc33 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.755632 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.756281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.756588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.756857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] PATH_INFO: `/v3/projects/f989212b8fa042d4b87343b405cb2925/users/71b1e67bd77940f8b14f0ded82f8848c/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.771592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f989212b8fa042d4b87343b405cb2925, user_id=71b1e67bd77940f8b14f0ded82f8848c, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.774253 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.789128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1160/2356] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:45 2022] PUT /identity/v3/projects/f989212b8fa042d4b87343b405cb2925/users/71b1e67bd77940f8b14f0ded82f8848c/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:45.794643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.805564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea279a-0bc9-42e5-98c4-cd5d3aad98e2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.865148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.865750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.866026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.866290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.866929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.868543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4307a73-03f1-4f70-b7ed-3656be3e34ae admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.878404 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1197/2357] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:45.883174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:45.893735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0bbd0-44bc-4d93-95f0-f964baa4f8ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:45.898545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:45.899182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:45.899456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:45.899721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] PATH_INFO: `/v3/projects/f989212b8fa042d4b87343b405cb2925/users/71b1e67bd77940f8b14f0ded82f8848c/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:45.901689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f989212b8fa042d4b87343b405cb2925, user_id=71b1e67bd77940f8b14f0ded82f8848c, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:45.904319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:45.914345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1161/2358] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:45 2022] PUT /identity/v3/projects/f989212b8fa042d4b87343b405cb2925/users/71b1e67bd77940f8b14f0ded82f8848c/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:23:46.541226 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:46.541601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:46.541937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:46.573298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] MFA Rules not processed for user `71b1e67bd77940f8b14f0ded82f8848c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:46.574556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:46.593552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:46.653247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1198/2359] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:47.116786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:47.118686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:47.196636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e6873-d760-4111-9275-0bacc3c00502 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:47.202493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:47.203206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:47.203480 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:47.203746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:47.205589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:47.218082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:47.219582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:47.229407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1162/2360] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:47.704451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:47.810268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803636c1-629c-4708-acc0-57fe15c1f4d6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:47.815462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:47.816087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:47.816377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:47.816657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] PATH_INFO: `/v3/users/2404b512ac7b4f69ad39d4e9c06e9ff2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:47.817826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2404b512ac7b4f69ad39d4e9c06e9ff2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:47.819773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:47.870664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2404b512ac7b4f69ad39d4e9c06e9ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:47.877608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2404b512ac7b4f69ad39d4e9c06e9ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:47.887884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:47.890433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2404b512ac7b4f69ad39d4e9c06e9ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:47.900577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2404b512ac7b4f69ad39d4e9c06e9ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:47.901200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2404b512ac7b4f69ad39d4e9c06e9ff2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:47.907262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1199/2361] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:23:47 2022] DELETE /identity/v3/users/2404b512ac7b4f69ad39d4e9c06e9ff2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:48.098180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:48.134525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45bb6-dec9-404c-933a-282911ba9285 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:48.163072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:48.195880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:48.196494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:48.196796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:48.197073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] PATH_INFO: `/v3/projects/f7b8f402cb2848219899c8f025ec39dd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:48.198228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b8f402cb2848219899c8f025ec39dd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:48.200132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:48.246724 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b8f402cb2848219899c8f025ec39dd', 'request_id': 'req-d651f788-a5b8-4732-8848-63e9cec8b461'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.252657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b8f402cb2848219899c8f025ec39dd', 'request_id': 'req-d651f788-a5b8-4732-8848-63e9cec8b461'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.258843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b8f402cb2848219899c8f025ec39dd', 'request_id': 'req-d651f788-a5b8-4732-8848-63e9cec8b461'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.285957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] The token cache is being invalidate because project f7b8f402cb2848219899c8f025ec39dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:23:48.286249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.286955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.288564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1163/2362] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:23:48 2022] DELETE /identity/v3/projects/f7b8f402cb2848219899c8f025ec39dd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:48.293148 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:48.294834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:48.319847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0248d-c76d-43fa-8fcd-ca483ed622e7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:48.390359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:48.391015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:48.391273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:48.391534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] PATH_INFO: `/v3/users/71b1e67bd77940f8b14f0ded82f8848c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:48.392894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b1e67bd77940f8b14f0ded82f8848c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:48.396613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:48.447665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b1e67bd77940f8b14f0ded82f8848c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.454787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b1e67bd77940f8b14f0ded82f8848c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.466816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b1e67bd77940f8b14f0ded82f8848c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.476591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b1e67bd77940f8b14f0ded82f8848c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.477406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b82592d-0c12-48b2-a971-d080bbc2186f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b1e67bd77940f8b14f0ded82f8848c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.483815 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1200/2363] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:23:48 2022] DELETE /identity/v3/users/71b1e67bd77940f8b14f0ded82f8848c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:48.667171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:48.669221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:48.682593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651f788-a5b8-4732-8848-63e9cec8b461 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:48.742512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:48.743147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:48.743423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:48.743698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] PATH_INFO: `/v3/projects/f989212b8fa042d4b87343b405cb2925` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:48.744877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f989212b8fa042d4b87343b405cb2925)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:48.746783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:48.795875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f989212b8fa042d4b87343b405cb2925', 'request_id': 'req-ae4dbb7f-e904-4d46-8806-14be41244540'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.797914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:48.802326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f989212b8fa042d4b87343b405cb2925', 'request_id': 'req-ae4dbb7f-e904-4d46-8806-14be41244540'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.809048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f989212b8fa042d4b87343b405cb2925', 'request_id': 'req-ae4dbb7f-e904-4d46-8806-14be41244540'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.823348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] The token cache is being invalidate because project f989212b8fa042d4b87343b405cb2925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:23:48.823629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.824299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae4dbb7f-e904-4d46-8806-14be41244540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:23:48.826196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1164/2364] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:23:48 2022] DELETE /identity/v3/projects/f989212b8fa042d4b87343b405cb2925 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:48.830963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:48.831456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:48.831553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:48.888789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:48.890000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:48.911857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:48.953713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1201/2365] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:48.961848 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:48.962241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:48.962588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:48.983688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:48.984950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:48.993532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1165/2366] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:48.998127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.008982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9db1c-2ea1-42e7-b09d-53e8d70888f2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.015480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.016119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.016397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.016716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.017370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:23:49.017761 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.019367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.031722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1202/2367] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:23:49.036303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.046531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee457a-4964-47bd-a0a4-d3635b2c0816 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.052388 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.053016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.053300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.053567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.054198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.056088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.086525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1166/2368] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:49.091006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.101355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25a409-bf0d-462d-8e27-049ee32fcaf4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.158523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.159136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.159416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.159680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.160359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.162368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.185847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1203/2369] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:49.190387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.201527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c910958-523d-4d7b-83e4-9a52cdfcb404 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.206732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.207330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.207643 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.207910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.208556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.210187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.220765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1167/2370] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:49.224950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.235266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f5384-2867-4e1f-9915-9bc70b358156 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.241151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.241740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.242013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.242276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.242916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.244580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b3042-7e98-46cb-ab6a-c1546a0cbbdd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.255235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1204/2371] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:49.259489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.269636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac7fcd-b2a2-4a9c-9271-ff00e8a67c9d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.274972 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.275653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.275956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.276264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] PATH_INFO: `/v3/projects/772d6d594e9b40e9912675060a886a38/users/dd08155021664188a97ee8696751294b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.292156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772d6d594e9b40e9912675060a886a38, user_id=dd08155021664188a97ee8696751294b, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.295498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.312429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1168/2372] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:49 2022] PUT /identity/v3/projects/772d6d594e9b40e9912675060a886a38/users/dd08155021664188a97ee8696751294b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:49.324096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.324379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.324679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.353197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] MFA Rules not processed for user `dd08155021664188a97ee8696751294b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:49.354431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:49.422274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1205/2373] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:49.428264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.491384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e92fa-de22-43d1-bdda-0c688691fb62 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.496746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.497342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.497620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.497905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.498528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.500420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.528718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1169/2374] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:23:49.532923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.546823 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8cb35-3f8d-45f3-8bb9-acf862c5aaab None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.567116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:49.607355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.607974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.608270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.608569 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.609243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.611185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.634847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1206/2375] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:23:49.639184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.650459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e48e-7f9f-4c02-808f-3d6e550ee272 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.656577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.657179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.657463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.657729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.658390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.660131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.671551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1170/2376] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:49.676356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.691355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99e07f-656e-4b63-9c8e-3b07948b92ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.696197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.696872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.697149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.697437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] PATH_INFO: `/v3/projects/c40d71d51a934b25a1d034e32600b3dd/users/e36917bfc60444c8a44eec4ea376dcd5/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.712271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40d71d51a934b25a1d034e32600b3dd, user_id=e36917bfc60444c8a44eec4ea376dcd5, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.714942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.729774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1207/2377] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:49 2022] PUT /identity/v3/projects/c40d71d51a934b25a1d034e32600b3dd/users/e36917bfc60444c8a44eec4ea376dcd5/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:49.733798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.743725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed065-37e6-4c4f-a38e-18a30c6399ac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.794285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:23:49.809085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.809997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.810377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.810825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.811831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.814404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a29575-b661-4590-88f3-38cb0db20d64 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.824952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1171/2378] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:23:49.829298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:49.840356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31261a-8e33-4516-aff0-1a8a309b00ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:49.845307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:49.845938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.846217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.846484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] PATH_INFO: `/v3/projects/c40d71d51a934b25a1d034e32600b3dd/users/e36917bfc60444c8a44eec4ea376dcd5/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.848476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40d71d51a934b25a1d034e32600b3dd, user_id=e36917bfc60444c8a44eec4ea376dcd5, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:49.851147 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:49.861740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1208/2379] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:23:49 2022] PUT /identity/v3/projects/c40d71d51a934b25a1d034e32600b3dd/users/e36917bfc60444c8a44eec4ea376dcd5/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:23:49.872019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:49.872303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:49.872606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:49.908141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] MFA Rules not processed for user `e36917bfc60444c8a44eec4ea376dcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:23:49.909435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:49.987036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1172/2380] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:23:50.001877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:50.003876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:23:50.081856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4d93-d8f7-4663-8aef-460d3a010687 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:50.086346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:50.086943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:50.087234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:50.087503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:50.098457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:50.099907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcab9c-d8ac-4d50-8702-7c37753e6b52 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:50.111064 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1209/2381] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:23:50.199245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:23:50.210242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dfe6e-8556-40e7-8105-8db3225ffcc3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:23:50.214365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:23:50.214994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:23:50.215277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:23:50.215548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:23:50.274865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:23:50.276257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:23:50.285082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1173/2382] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:00.105480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:00.281078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=68 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:09.673183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:09.673738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:09.673841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:09.694729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] MFA Rules not processed for user `38a331e431394e4882f31afd31f16d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:09.695913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:09.755806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1210/2383] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:19.748229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:35.430348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:35.431800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:35.530383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7539a4e-b9ea-4445-8fa3-cc0cf88c146a service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:35.535308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:35.536236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:35.536236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:35.536468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:35.538358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:35.550745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:35.552318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78259563-a44c-4dc4-8241-05ef2abd3e88 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:35.562428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1174/2384] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:37.616866 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:37.694423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7a99-05e5-4aca-ad57-81b5acfb2835 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:37.706734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:37.707552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:37.707930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:37.708298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] PATH_INFO: `/v3/users/dd08155021664188a97ee8696751294b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:37.710022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd08155021664188a97ee8696751294b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:37.712535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:37.805961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd08155021664188a97ee8696751294b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:37.814746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd08155021664188a97ee8696751294b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:37.830396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd08155021664188a97ee8696751294b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:37.842960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd08155021664188a97ee8696751294b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:37.843679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd08155021664188a97ee8696751294b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:37.861564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1211/2385] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:24:37 2022] DELETE /identity/v3/users/dd08155021664188a97ee8696751294b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:37.866339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:37.866840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:37.867095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:37.888884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:37.890189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:37.962768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1175/2386] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:24:37.972333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:37.974229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:38.063969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a59ae-79b7-4f0d-beea-ea2b028c1418 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:38.067638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:38.068245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:38.068883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:38.068883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:38.078302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:38.079747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:38.090711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1212/2387] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:38.389535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:38.391006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:38.414139 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642be332-6451-46b5-8a3a-da7cad82c726 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:38.421261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:38.421986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:38.422442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:38.423086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] PATH_INFO: `/v3/projects/772d6d594e9b40e9912675060a886a38` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:38.424988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=772d6d594e9b40e9912675060a886a38)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:38.427109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:38.520081 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772d6d594e9b40e9912675060a886a38', 'request_id': 'req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.535167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772d6d594e9b40e9912675060a886a38', 'request_id': 'req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.542510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772d6d594e9b40e9912675060a886a38', 'request_id': 'req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.545977 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:38.565286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] The token cache is being invalidate because project 772d6d594e9b40e9912675060a886a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:24:38.565570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.567140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.569461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1176/2388] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:24:38 2022] DELETE /identity/v3/projects/772d6d594e9b40e9912675060a886a38 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:38.574566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:38.582663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:38.591175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf7bc-49e8-4d34-bef2-d6a92455ba17 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:38.665115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:38.665813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:38.666028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:38.666299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] PATH_INFO: `/v3/users/e36917bfc60444c8a44eec4ea376dcd5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:38.667466 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36917bfc60444c8a44eec4ea376dcd5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:38.669819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:38.731630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36917bfc60444c8a44eec4ea376dcd5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.741444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36917bfc60444c8a44eec4ea376dcd5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.755591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36917bfc60444c8a44eec4ea376dcd5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.764701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36917bfc60444c8a44eec4ea376dcd5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.765501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb084f87-1397-4604-a389-2847a0d3fadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36917bfc60444c8a44eec4ea376dcd5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:38.772702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1213/2389] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:24:38 2022] DELETE /identity/v3/users/e36917bfc60444c8a44eec4ea376dcd5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:39.090527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.091989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:39.106426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fac5e6-3174-4ded-b5eb-ab7ee79f1ffa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.169765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.170502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.170794 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.171082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] PATH_INFO: `/v3/projects/c40d71d51a934b25a1d034e32600b3dd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.172722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40d71d51a934b25a1d034e32600b3dd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.174991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.227127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40d71d51a934b25a1d034e32600b3dd', 'request_id': 'req-9321aade-24f8-4257-9099-b7b42e99a185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:39.232776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40d71d51a934b25a1d034e32600b3dd', 'request_id': 'req-9321aade-24f8-4257-9099-b7b42e99a185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:39.238934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40d71d51a934b25a1d034e32600b3dd', 'request_id': 'req-9321aade-24f8-4257-9099-b7b42e99a185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:39.254425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] The token cache is being invalidate because project c40d71d51a934b25a1d034e32600b3dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:24:39.254716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:39.255576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9321aade-24f8-4257-9099-b7b42e99a185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:39.257304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1177/2390] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:24:39 2022] DELETE /identity/v3/projects/c40d71d51a934b25a1d034e32600b3dd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:39.262503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.262861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.263144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.283574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:39.284831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:39.346288 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1214/2391] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:39.355176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.355461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.355734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.375916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:39.376669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:39.387882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1178/2392] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:39.392464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.403631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0accf-2a0e-42b3-895e-b25718e3623c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.409503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.410117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.410393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.411126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.411330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:24:39.411632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.414187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.428068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1215/2393] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:24:39.433055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.442525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b6cb6-a4d2-49ed-8532-b178f8d13ba3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.447534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.448262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.448626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.448963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.449685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.451635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.483745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1179/2394] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:24:39.489422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.499751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6d63b-e846-459c-8bfc-4b6bdd0aa6f8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.504461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:39.562874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.563521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.563796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.564062 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.564739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.566686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.597094 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1216/2395] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:24:39.601261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.611997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82f52-92b8-4405-9914-e55b7866fd2a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.616653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.617232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.617508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.617843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.618567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.620172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.630203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1180/2396] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:39.634620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.644433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b7ac-961a-4698-abc1-62c90668f5d2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.651150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.652121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.652407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.652719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.653681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.655644 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e151b21-7bb3-404b-b932-d1293f8961ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.669519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1217/2397] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:39.674005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.687782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368707e-f607-43cd-bfce-498fe1e18b34 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.692905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.693566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.693815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.694123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] PATH_INFO: `/v3/projects/cc6734d54d8c4514bbeaa0fd8084ac67/users/f4c0bece1c0e4f0689a9c03d8eb2a75b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.709863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6734d54d8c4514bbeaa0fd8084ac67, user_id=f4c0bece1c0e4f0689a9c03d8eb2a75b, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.712628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c68f33-2106-497d-a1e6-554dc6c236d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.729798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1181/2398] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:24:39 2022] PUT /identity/v3/projects/cc6734d54d8c4514bbeaa0fd8084ac67/users/f4c0bece1c0e4f0689a9c03d8eb2a75b/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:39.740710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.740943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.741115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.769630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] MFA Rules not processed for user `f4c0bece1c0e4f0689a9c03d8eb2a75b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:39.770838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:39.829424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.830245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.831028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.842047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1218/2399] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:39.846234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.861678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] MFA Rules not processed for user `ed4711b7691643109007e173fee1c405`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:39.862930 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:39.879880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:39.917675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6df9-a43f-4f8f-a553-fd9047cd623d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:39.923769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:39.924655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:39.924940 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:39.925215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:39.925972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:39.928031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:39.930358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1182/2400] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:39.958112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1219/2401] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:24:39.962290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:39.973481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb6a20-0555-4a99-a9ba-d16762e217b1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.030055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.030804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.031085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.031360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.031993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.033942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.057843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1183/2402] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:24:39 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:24:40.062110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.067693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:40.073584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a32d-4f2e-4bd7-9265-e02230381d65 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.079485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.080278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.080591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.080850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.081498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.083102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.093409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1220/2403] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:40.097735 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.108204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb532b5-589f-4340-9460-fbc0067492ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.113987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.114630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.114903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.115171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] PATH_INFO: `/v3/projects/231a32f281ce41f48d4185ab6ab201a5/users/4003bce0bf624b35a041950cb5325c99/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.131469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a32f281ce41f48d4185ab6ab201a5, user_id=4003bce0bf624b35a041950cb5325c99, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.134311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.149921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1184/2404] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:24:40 2022] PUT /identity/v3/projects/231a32f281ce41f48d4185ab6ab201a5/users/4003bce0bf624b35a041950cb5325c99/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:24:40.153886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.164302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfac2-a204-492b-af15-fa1082d13e52 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.224737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.225364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.225645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.225918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.226566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.228296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3669-328a-4121-9c48-123ee72eb632 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.238703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1221/2405] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:40.243007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.253695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025679fc-b573-4f1e-92b6-7dfa1c1e7261 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.258325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.259033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.259311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.259587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] PATH_INFO: `/v3/projects/231a32f281ce41f48d4185ab6ab201a5/users/4003bce0bf624b35a041950cb5325c99/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.261572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a32f281ce41f48d4185ab6ab201a5, user_id=4003bce0bf624b35a041950cb5325c99, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.264185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.274812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1185/2406] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:24:40 2022] PUT /identity/v3/projects/231a32f281ce41f48d4185ab6ab201a5/users/4003bce0bf624b35a041950cb5325c99/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:40.286303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.286704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.287075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.317018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] MFA Rules not processed for user `4003bce0bf624b35a041950cb5325c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:40.318300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:40.389105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1222/2407] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:24:40.400352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.402102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:40.486272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcb9b-be85-4e0b-ad7c-72b606bc5e74 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.490718 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.491492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.491774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.492045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.501952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.503421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75989977-914c-48ae-8cd4-a2132f58f1ef service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.514647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1186/2408] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:40.521259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:40.675530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.738696 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96078b-8d0d-47b7-8513-7d5efe5ead47 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:40.744096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:40.744821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.745212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.745474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] PATH_INFO: `/v3/users/f4c0bece1c0e4f0689a9c03d8eb2a75b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.746624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c0bece1c0e4f0689a9c03d8eb2a75b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:40.748591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:40.763770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:40.801327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c0bece1c0e4f0689a9c03d8eb2a75b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:40.808666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c0bece1c0e4f0689a9c03d8eb2a75b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:40.821884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c0bece1c0e4f0689a9c03d8eb2a75b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:40.830474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c0bece1c0e4f0689a9c03d8eb2a75b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:40.831281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c0bece1c0e4f0689a9c03d8eb2a75b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:40.838584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1223/2409] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:24:40 2022] DELETE /identity/v3/users/f4c0bece1c0e4f0689a9c03d8eb2a75b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:40.843094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:40.843378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:40.843645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:40.862298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:40.863513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:40.926595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1187/2410] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:40.936811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:40.938532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:41.002283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db8a16-ad90-4804-b21f-5b852073a600 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:41.006009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:41.006628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:41.006907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:41.007179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:41.016745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:41.018137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:41.028789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1224/2411] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:41.313285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:41.317609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:41.332291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3a224-bb0b-4e95-b8bb-780c2c2b218b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:41.337932 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:41.338730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:41.339137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:41.339513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] PATH_INFO: `/v3/projects/cc6734d54d8c4514bbeaa0fd8084ac67` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:41.340929 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6734d54d8c4514bbeaa0fd8084ac67)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:41.343223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:41.395376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6734d54d8c4514bbeaa0fd8084ac67', 'request_id': 'req-79c8102a-17fd-41ab-9f04-91060c0bc1cd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.401665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6734d54d8c4514bbeaa0fd8084ac67', 'request_id': 'req-79c8102a-17fd-41ab-9f04-91060c0bc1cd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.408183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6734d54d8c4514bbeaa0fd8084ac67', 'request_id': 'req-79c8102a-17fd-41ab-9f04-91060c0bc1cd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.423902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] The token cache is being invalidate because project cc6734d54d8c4514bbeaa0fd8084ac67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:24:41.424369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.425191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.426741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1188/2412] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:24:41 2022] DELETE /identity/v3/projects/cc6734d54d8c4514bbeaa0fd8084ac67 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:41.436261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:41.446280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29525d-b916-4f12-aaa4-0ec1d97b2642 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:41.502273 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:41.502908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:41.503186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:41.503468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] PATH_INFO: `/v3/users/4003bce0bf624b35a041950cb5325c99` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:41.504713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4003bce0bf624b35a041950cb5325c99)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:41.506665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:41.549327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:41.558057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4003bce0bf624b35a041950cb5325c99'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.565314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4003bce0bf624b35a041950cb5325c99'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.582189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4003bce0bf624b35a041950cb5325c99'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.593371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4003bce0bf624b35a041950cb5325c99'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.594200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b15b6ade-5051-49c9-b43d-dc334048bdb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4003bce0bf624b35a041950cb5325c99'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:41.601131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1225/2413] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:24:41 2022] DELETE /identity/v3/users/4003bce0bf624b35a041950cb5325c99 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:41.959192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:41.960805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:41.974369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8102a-17fd-41ab-9f04-91060c0bc1cd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.051436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.052122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.052399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.052714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] PATH_INFO: `/v3/projects/231a32f281ce41f48d4185ab6ab201a5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.053896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=231a32f281ce41f48d4185ab6ab201a5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.055868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.087299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:42.104609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231a32f281ce41f48d4185ab6ab201a5', 'request_id': 'req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:42.110020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231a32f281ce41f48d4185ab6ab201a5', 'request_id': 'req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:42.116141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231a32f281ce41f48d4185ab6ab201a5', 'request_id': 'req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:42.130263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] The token cache is being invalidate because project 231a32f281ce41f48d4185ab6ab201a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:24:42.130527 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:42.131226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c5a11-3ea1-4ad4-bd39-645d8708a69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:42.132783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1189/2414] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:24:41 2022] DELETE /identity/v3/projects/231a32f281ce41f48d4185ab6ab201a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:42.137944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.138444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.138539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.160140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:42.161405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:42.222801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1226/2415] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:42.230737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.231054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.231263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.252981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:42.254285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:42.263106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1190/2416] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:42.268124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.279278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1925d-9839-4fb6-a102-1f1f872e22e8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.284479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.285120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.285397 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.285729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.286386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:24:42.286874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.288532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.302562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1227/2417] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:24:42.307203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.316834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4ba4-3106-47e4-b1a5-21db5772d04f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.321906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.322614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.323230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.323593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.324433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.326348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.354466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1191/2418] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:24:42 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:24:42.358594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.368687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45a79-b21c-45a3-9fb4-904c27bbe9ae admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.426771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.427380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.427656 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.427923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.428623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.430573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.457441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1228/2419] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:24:42 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:24:42.460064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.471397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d7-be8e-48d3-9f71-7df8f2057d81 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.476817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.477650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.477931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.478667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.478855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.480835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.492013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1192/2420] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:42.496087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.506732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55482133-b077-4ce4-ad44-c510e9e74d4b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.511427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.512031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.512308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.512599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.513244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.514950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb5d37-8117-43b0-ae0c-8c6aecdf997a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.527138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1229/2421] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:42.532546 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.545374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df64b8c-95ed-472e-9974-1db02c7a6c7c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.550607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.551396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.551746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.552095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] PATH_INFO: `/v3/projects/d37f6ea1ab624038b2290b3497c915b8/users/ea7c0a282b394faaab93701e644d6041/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.568462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37f6ea1ab624038b2290b3497c915b8, user_id=ea7c0a282b394faaab93701e644d6041, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.571476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.592354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1193/2422] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:24:42 2022] PUT /identity/v3/projects/d37f6ea1ab624038b2290b3497c915b8/users/ea7c0a282b394faaab93701e644d6041/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:42.603043 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.603703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.603703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.636557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] MFA Rules not processed for user `ea7c0a282b394faaab93701e644d6041`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:42.638146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:42.647878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:42.723060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1230/2423] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:42.763701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:42.765863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:42.841820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a40500-87a6-4528-9770-8c3138394e6d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:42.859656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:42.860887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:42.860887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:42.861091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:42.884172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:42.885702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcec409-0adb-4535-abbe-2e4b8b3a1eb3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:42.905748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1194/2424] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:42.922579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:43.016168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd0c9d-4249-4d5f-9775-d8c844b6495c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:43.021777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:43.022607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.022952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.023984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] PATH_INFO: `/v3/users/ea7c0a282b394faaab93701e644d6041` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.024980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7c0a282b394faaab93701e644d6041)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:43.027306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:43.082636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7c0a282b394faaab93701e644d6041'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.090097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7c0a282b394faaab93701e644d6041'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.102310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7c0a282b394faaab93701e644d6041'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.110737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7c0a282b394faaab93701e644d6041'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.111538 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7c0a282b394faaab93701e644d6041'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.117602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1231/2425] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:24:42 2022] DELETE /identity/v3/users/ea7c0a282b394faaab93701e644d6041 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:43.122375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.122673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.122950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.142561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:43.143758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:43.192407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:43.207171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1195/2426] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:43.217111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:43.218655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:43.251358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:43.279363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977409d6-3a67-40ed-a750-f5e12b6fb865 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:43.283191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:43.283803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.284080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.285949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.296229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:43.297683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fb30f-8774-40df-a992-557080e8ae35 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:43.310547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1232/2427] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:24:43.688226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:43.689957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:43.710203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fcb08-ff36-449e-89e7-f101bce484dc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:43.715675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:43.716334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.716653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.716955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] PATH_INFO: `/v3/projects/d37f6ea1ab624038b2290b3497c915b8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.718097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37f6ea1ab624038b2290b3497c915b8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:43.719964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:43.772895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37f6ea1ab624038b2290b3497c915b8', 'request_id': 'req-e4962acd-e055-42be-b2ab-47258bd074d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.779024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37f6ea1ab624038b2290b3497c915b8', 'request_id': 'req-e4962acd-e055-42be-b2ab-47258bd074d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.785318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37f6ea1ab624038b2290b3497c915b8', 'request_id': 'req-e4962acd-e055-42be-b2ab-47258bd074d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.799304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] The token cache is being invalidate because project d37f6ea1ab624038b2290b3497c915b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:24:43.799597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.800239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4962acd-e055-42be-b2ab-47258bd074d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:24:43.802039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1196/2428] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:24:43 2022] DELETE /identity/v3/projects/d37f6ea1ab624038b2290b3497c915b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:43.806932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.807302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.807599 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.827223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:43.828431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:43.891142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1233/2429] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:24:43.900147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.900433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.900733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.922217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:43.923595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:43.932057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1197/2430] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:43.936371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:43.946271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461cf6d-cdd1-449c-998d-2590f5db5289 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:43.950955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:43.951596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:43.951870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:43.952138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:43.952834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:24:43.953137 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:43.954820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:43.966751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1234/2431] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:24:44.030186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.039534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5882e2-2193-4cc2-b62c-ccdfe1bec711 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.044874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.045468 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.045761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.046022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.046671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.048555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.078142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1198/2432] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:24:44 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:24:44.082272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.092528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dc3ba-ce63-474d-ae49-03e2273e86c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.150364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.150978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.151276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.151540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.152270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.154312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.180624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1235/2433] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:24:44 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:24:44.184858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.195471 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d00ba-4097-4e97-af20-14f8e1d2d416 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.200232 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.200858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.201141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.201418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.202102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.203719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.214072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1199/2434] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:44.218000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.228979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333ea6-9375-4d35-8f5f-f541386f4fe0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.234666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.235384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.235661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.235931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.236615 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.238236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d0b25-a362-42ea-bcb5-2abe5d59cfdd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.253989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1236/2435] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:24:44.260235 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.274775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1243d5-95f6-4c94-acb8-002f280a2677 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.279904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.280586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.280868 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.281140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] PATH_INFO: `/v3/projects/75e24ec2c109424ab964b4d441db44e7/users/b661e219b74f436bb668806a547f4778/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.296465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e24ec2c109424ab964b4d441db44e7, user_id=b661e219b74f436bb668806a547f4778, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.299156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.314920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1200/2436] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:24:44 2022] PUT /identity/v3/projects/75e24ec2c109424ab964b4d441db44e7/users/b661e219b74f436bb668806a547f4778/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:24:44.319945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1eff7-13c9-4da5-bdae-e2b420ba304c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.320213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1eff7-13c9-4da5-bdae-e2b420ba304c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.320479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1eff7-13c9-4da5-bdae-e2b420ba304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.339989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5fa1eff7-13c9-4da5-bdae-e2b420ba304c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:44.341219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5fa1eff7-13c9-4da5-bdae-e2b420ba304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:44.373607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:44.403136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1237/2437] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:44.412985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:44.414839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:44.473941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175aea-b8cc-4217-9227-74eedd2a3728 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:44.478782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:44.479954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:44.480487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:44.480793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:44.490897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:44.492401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:44.502320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1201/2438] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:24:45.248919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62a521-00e8-43ee-95aa-87bbfa1981c3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:45.249452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62a521-00e8-43ee-95aa-87bbfa1981c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:45.249452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62a521-00e8-43ee-95aa-87bbfa1981c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:45.282145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0a62a521-00e8-43ee-95aa-87bbfa1981c3 None None] MFA Rules not processed for user `b661e219b74f436bb668806a547f4778`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:45.283594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0a62a521-00e8-43ee-95aa-87bbfa1981c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:45.356149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1238/2439] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:45.367379 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:45.368928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:45.388432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:45.430025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4573d-004f-4161-bcfd-50b82200b547 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:45.434187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:45.434817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:45.435079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:45.435495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:45.444323 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:45.445796 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e751-11bc-4de6-9760-6736ebc17e03 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:45.454807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1202/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:24:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:24:46.828480 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:46.829013 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:46.829109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:46.849134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] MFA Rules not processed for user `b661e219b74f436bb668806a547f4778`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:46.850337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:46.858861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1239/2441] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:24:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:48.795618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9828e-a7ee-4dab-b66a-7edf69a7ec19 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:48.796150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9828e-a7ee-4dab-b66a-7edf69a7ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:48.796757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9828e-a7ee-4dab-b66a-7edf69a7ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:48.830776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ccf9828e-a7ee-4dab-b66a-7edf69a7ec19 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:24:48.832247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ccf9828e-a7ee-4dab-b66a-7edf69a7ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:48.910757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1203/2442] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:24:48.963427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:24:48.965862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:24:49.048217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e01c5-e31c-47f5-aa74-9265d3a306cb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:24:49.053781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:24:49.054414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:24:49.054890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:24:49.055302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:24:49.064263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:24:49.065901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58994255-3187-4872-9823-cffe46c24b39 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:24:49.074579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1240/2443] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:24:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:24:58.904097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=51 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:24:59.071919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=77 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:06.477774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f54a37-5f60-4d43-875b-cde7e440225b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.478358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f54a37-5f60-4d43-875b-cde7e440225b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.478358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f54a37-5f60-4d43-875b-cde7e440225b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.500840 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-55f54a37-5f60-4d43-875b-cde7e440225b None None] MFA Rules not processed for user `b661e219b74f436bb668806a547f4778`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:06.502125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55f54a37-5f60-4d43-875b-cde7e440225b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:06.510156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1204/2444] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:25:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:25:06.519910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.520380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.520666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.522378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1241/2445] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:25:06.528641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504ce5-32be-4e5e-8d49-c1d518990d8b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.528928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504ce5-32be-4e5e-8d49-c1d518990d8b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.529199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87504ce5-32be-4e5e-8d49-c1d518990d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.566733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-87504ce5-32be-4e5e-8d49-c1d518990d8b None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:06.567320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-87504ce5-32be-4e5e-8d49-c1d518990d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:06.645266 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1205/2446] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:25:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:06.657004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:06.658788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:06.722079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d508a17-3992-40d8-b70a-be5838ffe33b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:06.726046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:06.726763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.727253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.727586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.743406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:06.744985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db744231-898d-4305-9996-3c9dcb137730 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:06.756193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1242/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:25:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:25:06.827082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.827652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.827926 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.829344 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1206/2448] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:25:06.834089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66892761-8443-45a6-8f63-a41c669bb33c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.834974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66892761-8443-45a6-8f63-a41c669bb33c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.835252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66892761-8443-45a6-8f63-a41c669bb33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.858694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-66892761-8443-45a6-8f63-a41c669bb33c None None] MFA Rules not processed for user `7f71b5efa95b41e196a505798527904d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:06.860723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-66892761-8443-45a6-8f63-a41c669bb33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:06.867948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1243/2449] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:25:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:06.881599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:06.883753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:06.897835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bdde0-1012-443d-aac8-9e06fc27bab0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:06.901441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:06.902053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:06.902340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:06.902664 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:06.910397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:06.911767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53db0c3-11b8-479e-8c5e-87a8f1336d2c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:06.920599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1207/2450] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:25:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:25:07.004104 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98980a9-7816-4574-a508-9e831af37be7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:07.004454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98980a9-7816-4574-a508-9e831af37be7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:07.004890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98980a9-7816-4574-a508-9e831af37be7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:07.006479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1244/2451] 10.222.0.40 () {54 vars in 921 bytes} [Sun Aug 28 10:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 28 10:25:07.012286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1bd4-3482-4102-af22-39efc50aa3c5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:07.012725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1bd4-3482-4102-af22-39efc50aa3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:07.013060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1bd4-3482-4102-af22-39efc50aa3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:07.032899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-05dc1bd4-3482-4102-af22-39efc50aa3c5 None None] MFA Rules not processed for user `cbe80d5de4bb4746a1756d1077e647a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:07.034106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-05dc1bd4-3482-4102-af22-39efc50aa3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:07.043352 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1208/2452] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Aug 28 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:25:07.055003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98980a9-7816-4574-a508-9e831af37be7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:07.067120 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98980a9-7816-4574-a508-9e831af37be7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:07.070898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:07.071501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:07.071820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:07.072131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:07.083019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:07.085479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:07.097622 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1245/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:25:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:25:09.995210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813970b3-037b-4964-9fcd-41c5d2b879ee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:09.995725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813970b3-037b-4964-9fcd-41c5d2b879ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:09.995825 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813970b3-037b-4964-9fcd-41c5d2b879ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:10.016374 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-813970b3-037b-4964-9fcd-41c5d2b879ee None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:10.017641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-813970b3-037b-4964-9fcd-41c5d2b879ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:10.025814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1209/2454] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:10.037807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:10.050027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d373-88d3-4aac-a5e4-b4bb8d3d0daa service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:10.055405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:10.056081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:10.056424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:10.056805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:10.065539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:10.067244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:10.076609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1246/2455] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:25:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:25:20.011309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=55 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:20.072677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=41 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:29.538725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391899-898d-4ec7-b747-d229b668ae87 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:29.539211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391899-898d-4ec7-b747-d229b668ae87 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:29.539211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391899-898d-4ec7-b747-d229b668ae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:29.560272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4f391899-898d-4ec7-b747-d229b668ae87 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:29.561531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4f391899-898d-4ec7-b747-d229b668ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:29.569733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1210/2456] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:29.582717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:29.594287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc13959-f687-41bc-984d-5c2a7809cfcc service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:29.600995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:29.601620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:29.601955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:29.602168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:29.612021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:29.613313 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:29.622877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1247/2457] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:25:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:25:39.555128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:39.618647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:43.754642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:43.755107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:43.755223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:43.777593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:43.779424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:43.792121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1211/2458] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:43.804795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:43.816012 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7308-0d96-4bb4-a3f4-b5abda11dcad service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:43.822108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:43.822704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:43.822982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:43.823285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:43.833406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:43.835914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:43.849008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1248/2459] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:25:46.536993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:46.549159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1cde3-2bfe-4f35-94fe-dd453d6e170d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:46.555518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:46.556101 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:46.556380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:46.556680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:46.558489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:46.576454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:46.578411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0202-ea09-43c4-a6e8-331acad560db service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:46.603186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1212/2460] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:25:47.819900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:47.831378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d7387-f80e-4102-981d-346d06cb49ab service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:47.835980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:47.837213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:47.837746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:47.838253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:47.840261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:47.856150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:47.858393 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fdece-3eb3-4a7d-ac8f-26b67e8f9ecd service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:47.873157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1249/2461] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:25:50.251271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:50.252339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:50.252749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:50.273769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] MFA Rules not processed for user `3deb7adb33004f1097574470754611ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:50.275137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:50.350649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1213/2462] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:57.868867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=12 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:25:59.027930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:59.028494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:59.028494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:59.048528 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:25:59.049750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:25:59.058106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1250/2463] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:25:59.070417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:25:59.083411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a285269-13b3-4fc5-84b9-6621c6e86cd2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:25:59.089422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:25:59.090029 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:25:59.090309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:25:59.090589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:25:59.098975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:25:59.100556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:25:59.109024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1214/2464] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:26:01.069112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.081362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e07d7-5585-416f-9bf3-ec33cb4bd368 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.086657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.087366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.087653 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.087922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] PATH_INFO: `/v3/users/b661e219b74f436bb668806a547f4778` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.089074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b661e219b74f436bb668806a547f4778)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.091009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.138784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b661e219b74f436bb668806a547f4778'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.145473 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b661e219b74f436bb668806a547f4778'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.159381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b661e219b74f436bb668806a547f4778'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.172713 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b661e219b74f436bb668806a547f4778'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.173339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3db64191-f79c-43e2-b3f5-0fcbd3749eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b661e219b74f436bb668806a547f4778'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.181500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1251/2465] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:26:01 2022] DELETE /identity/v3/users/b661e219b74f436bb668806a547f4778 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:01.362325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.375414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a32794-e94e-4f77-8150-5b61bfe540e1 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.435265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.435864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.436138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.436404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] PATH_INFO: `/v3/projects/75e24ec2c109424ab964b4d441db44e7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.437555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e24ec2c109424ab964b4d441db44e7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.439457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.486620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e24ec2c109424ab964b4d441db44e7', 'request_id': 'req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.492099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e24ec2c109424ab964b4d441db44e7', 'request_id': 'req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.498071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e24ec2c109424ab964b4d441db44e7', 'request_id': 'req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.512678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] The token cache is being invalidate because project 75e24ec2c109424ab964b4d441db44e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:26:01.512967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.513579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b64871c-8ebc-4f11-994c-a955b5eb7eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:01.515142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1215/2466] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:26:01 2022] DELETE /identity/v3/projects/75e24ec2c109424ab964b4d441db44e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:01.520381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.520710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.520980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.541566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:01.542717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:01.610756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1252/2467] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:01.626836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.627157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.627558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.638490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:01.653086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:01.654362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:01.662896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1216/2468] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:01.667287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.677581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc870f8a-4cf2-48df-913d-a1f8face16c3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.682697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.683300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.683567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.683839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.684492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:26:01.684816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.686427 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.698152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1253/2469] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:26:01.702368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.712191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cb5-93ba-40e3-9977-73491692e6e5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.717880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.718503 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.718783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.719048 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.719681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.721631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.766745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1217/2470] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:01 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:01.772657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.783260 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0d35b-bb8a-42bd-862d-f4666702c1ce admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.831044 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:01.841085 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.841918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.842208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.842747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.843407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.845570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.869096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1254/2471] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:01 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:01.873537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.884635 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c571-067c-49fc-ab66-de8d688d411d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.890450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.891030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.891334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.891599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.892230 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.893906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.904268 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1218/2472] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:01.908283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.918889 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e8a26-af2e-4ef2-bfaf-082a8dda4df4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.924619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.925214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.925491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.925765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.926411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.928029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f5f8d-1c91-4d7c-928b-8679c3290a0d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.938835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1255/2473] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:01.942832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:01.953368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76e16-d6ff-4a86-8590-e4faa5a02626 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:01.960110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:01.960803 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:01.961078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:01.961348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] PATH_INFO: `/v3/projects/805bd1eda126437d9c2997927d9da8c3/users/ec6cbf7f279a4b51994ec4508be71668/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:01.977009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=805bd1eda126437d9c2997927d9da8c3, user_id=ec6cbf7f279a4b51994ec4508be71668, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:01.979774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:01.994554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1219/2474] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:01 2022] PUT /identity/v3/projects/805bd1eda126437d9c2997927d9da8c3/users/ec6cbf7f279a4b51994ec4508be71668/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:01.999742 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd882-c140-41fb-b182-b151e500c4a2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:02.000011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd882-c140-41fb-b182-b151e500c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:02.000282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd882-c140-41fb-b182-b151e500c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:02.020018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-eeedd882-c140-41fb-b182-b151e500c4a2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:02.021267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eeedd882-c140-41fb-b182-b151e500c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:02.082997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1256/2475] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:02.093190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:02.095228 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:02.155711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d949f-9fbe-4b9b-9bbc-8301602ecd1b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:02.161582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:02.162514 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:02.162982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:02.163269 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:02.172916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:02.174350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:02.185207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1220/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:06.136674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.137585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.137929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.171181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] MFA Rules not processed for user `ec6cbf7f279a4b51994ec4508be71668`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:06.172728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:06.262530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1257/2477] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:06.266741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.280812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f58f-ecf2-4845-954c-28aae659fa69 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.285262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.285857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.286131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.286400 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.287025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.289138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.319165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1221/2478] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:06.323121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.334198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d541b9-85e7-4de8-b103-390f9b18d8e6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.392964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.393520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.393800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.394074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.394864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.396900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.413502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:06.429121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1258/2479] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:06 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:06.431357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.442517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f3e9-c7db-411d-b5d4-f4b8e146fd97 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.448156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.448812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.449094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.449367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.450079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.451698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.463981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1222/2480] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:06.468362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.480451 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efefbd3-e09e-41ea-827b-e1c04de28efa admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.485336 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.486007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.486281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.486548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] PATH_INFO: `/v3/projects/1a9fe9034993482faa627a742fecc63f/users/888e69fe4f9d4f7291c275382b918460/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.502169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9fe9034993482faa627a742fecc63f, user_id=888e69fe4f9d4f7291c275382b918460, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.505040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.521264 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1259/2481] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:06 2022] PUT /identity/v3/projects/1a9fe9034993482faa627a742fecc63f/users/888e69fe4f9d4f7291c275382b918460/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:06.524986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.535254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d288d-8a8e-49aa-a459-0dc72c6ac808 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.612804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.613634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.614403 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.614841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.615456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.617426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e1c33-45de-4085-b7e8-c8f2bcdcf13f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.618981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:06.633549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1223/2482] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:06.634110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:06.645468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1aa82-aca5-4b67-b8a9-fa596e6c523d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:06.650325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:06.651073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:06.651408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:06.651740 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] PATH_INFO: `/v3/projects/1a9fe9034993482faa627a742fecc63f/users/888e69fe4f9d4f7291c275382b918460/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:06.653738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9fe9034993482faa627a742fecc63f, user_id=888e69fe4f9d4f7291c275382b918460, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:06.656488 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:06.667765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1260/2483] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:06 2022] PUT /identity/v3/projects/1a9fe9034993482faa627a742fecc63f/users/888e69fe4f9d4f7291c275382b918460/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:26:11.771010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4862f-7468-4a06-b3b1-c711ec147651 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:11.771512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4862f-7468-4a06-b3b1-c711ec147651 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:11.771612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4862f-7468-4a06-b3b1-c711ec147651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:11.821452 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-39c4862f-7468-4a06-b3b1-c711ec147651 None None] MFA Rules not processed for user `888e69fe4f9d4f7291c275382b918460`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:11.822555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-39c4862f-7468-4a06-b3b1-c711ec147651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:11.920304 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1224/2484] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:11.934257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:11.936919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:12.044233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdd29-1c0a-4034-bbb4-2849fc420333 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:12.053365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:12.054587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:12.055495 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:12.056819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:12.137728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:12.141105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:12.153802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1261/2485] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:13.278771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:13.279317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:13.279444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:13.300439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] MFA Rules not processed for user `ec6cbf7f279a4b51994ec4508be71668`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:13.301689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:13.309363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1225/2486] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:22.148311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=4,keystone:SELECT=73 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:23.295378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=39 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:44.400402 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:44.403150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:44.482726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7724af-6586-4dbd-96c1-fda6d98d658b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:44.490635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:44.491733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:44.492189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:44.492662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:44.495374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:44.511803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:44.513691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:44.528631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1262/2487] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:50.843577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:50.855771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fc28-eaf8-4bed-83eb-2920af81a296 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:50.860265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:50.861111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:50.861459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:50.861827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:50.863716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:50.937645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:50.939194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:50.949234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1226/2488] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:51.433058 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:51.450270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce5cde-3e58-4248-a030-a89f673542c1 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:51.458054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:51.459471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:51.459883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:51.460005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:51.465118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:51.488099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:51.489623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:51.502505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1263/2489] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:55.316398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:55.327297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816df40-cb2f-4931-8b4f-81264ddc9f2d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:55.331273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:55.331896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:55.332331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:55.332683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:55.335597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:55.349771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:55.352695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:55.364997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1227/2490] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:55.679546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:55.681090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:55.701051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d77b3-3af5-4fe6-90f1-25be3815b1d3 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:55.707968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:55.710103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:55.721517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:55.722183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:55.722460 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:55.722732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] PATH_INFO: `/v3/users/38a331e431394e4882f31afd31f16d46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:55.724390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a331e431394e4882f31afd31f16d46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:55.726817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:55.730824 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500729e2-c4e5-4e08-9dfd-adda67ca311d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:55.737710 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:55.739843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:55.740197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:55.740575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] PATH_INFO: `/v3/users/ed4711b7691643109007e173fee1c405` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:55.762075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4711b7691643109007e173fee1c405)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:55.764225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:55.804326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a331e431394e4882f31afd31f16d46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.815720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a331e431394e4882f31afd31f16d46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.832363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a331e431394e4882f31afd31f16d46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.840967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4711b7691643109007e173fee1c405'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.845298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a331e431394e4882f31afd31f16d46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.846687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a331e431394e4882f31afd31f16d46'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.850931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4711b7691643109007e173fee1c405'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.851894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4711b7691643109007e173fee1c405'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.860640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1264/2491] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:26:55 2022] DELETE /identity/v3/users/38a331e431394e4882f31afd31f16d46 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:55.868001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4711b7691643109007e173fee1c405'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.868802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4711b7691643109007e173fee1c405'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:55.905533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1228/2492] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:26:55 2022] DELETE /identity/v3/users/ed4711b7691643109007e173fee1c405 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:56.078807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.080719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.171614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75a040-4d97-4007-897b-dd48b5946378 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.178311 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:56.262941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.266848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.269888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.270301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] PATH_INFO: `/v3/projects/0e47be8f2ac14c19ab40e293422c845d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.271877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e47be8f2ac14c19ab40e293422c845d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.274820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.289138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.290729 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.305106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62921411-6e1e-4baa-9c30-30cc84f5d55d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.310347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.310985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.311498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.312003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] PATH_INFO: `/v3/projects/eefcb341ab8d4823a95c5bce47395083` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.335190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eefcb341ab8d4823a95c5bce47395083)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.339178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.352130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e47be8f2ac14c19ab40e293422c845d', 'request_id': 'req-3a3b6738-7634-4ed9-b003-1afc0cd0278a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.364746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e47be8f2ac14c19ab40e293422c845d', 'request_id': 'req-3a3b6738-7634-4ed9-b003-1afc0cd0278a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.375222 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e47be8f2ac14c19ab40e293422c845d', 'request_id': 'req-3a3b6738-7634-4ed9-b003-1afc0cd0278a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.388626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] The token cache is being invalidate because project 0e47be8f2ac14c19ab40e293422c845d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:26:56.388983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.389881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3a3b6738-7634-4ed9-b003-1afc0cd0278a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.391318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1265/2493] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:26:56 2022] DELETE /identity/v3/projects/0e47be8f2ac14c19ab40e293422c845d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:56.397811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce2a5-b71c-4076-bfe3-7d81eae61c93 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.398093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce2a5-b71c-4076-bfe3-7d81eae61c93 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.398365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce2a5-b71c-4076-bfe3-7d81eae61c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.400389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:56.421045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eefcb341ab8d4823a95c5bce47395083', 'request_id': 'req-df5c6034-cf72-4f22-9a8c-3c86b023eca1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.423176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1a3ce2a5-b71c-4076-bfe3-7d81eae61c93 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.424441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a3ce2a5-b71c-4076-bfe3-7d81eae61c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.427458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eefcb341ab8d4823a95c5bce47395083', 'request_id': 'req-df5c6034-cf72-4f22-9a8c-3c86b023eca1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.435935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eefcb341ab8d4823a95c5bce47395083', 'request_id': 'req-df5c6034-cf72-4f22-9a8c-3c86b023eca1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.454765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] The token cache is being invalidate because project eefcb341ab8d4823a95c5bce47395083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:26:56.455231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.456120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-df5c6034-cf72-4f22-9a8c-3c86b023eca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:56.458088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1229/2494] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:26:56 2022] DELETE /identity/v3/projects/eefcb341ab8d4823a95c5bce47395083 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:56.462614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69982b9-51a1-4f5b-8b07-d8ca36e41923 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.462905 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69982b9-51a1-4f5b-8b07-d8ca36e41923 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.463181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69982b9-51a1-4f5b-8b07-d8ca36e41923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.486519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a69982b9-51a1-4f5b-8b07-d8ca36e41923 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.487964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a69982b9-51a1-4f5b-8b07-d8ca36e41923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.493961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1266/2495] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:56.501999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.502282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.502557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.503103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1230/2496] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:56.510799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.511281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.511555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.524579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.525807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.532999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.534172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.535159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1267/2497] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:56.538968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.545667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1231/2498] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:56.549875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.551041 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071bce1-afb9-4694-98fe-b3a9b3c7b77d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.556225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.556881 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.557163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.557429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.558093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:26:56.558396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.559671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab42aa-0adc-4be5-9670-627ff26a57ce None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.560165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.565005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.565676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.565956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.566231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.566889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:26:56.567218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.568836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.576549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1268/2499] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:26:56.579155 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.581690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1232/2500] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:26:56.586047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.595278 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af55945-4f04-4dd4-82c5-60ad2acc25ba admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.600699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.601317 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.601594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.601860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.602257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14fc-cbf8-4999-9e98-6bf73daa0de3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.602497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.604416 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.607696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.608392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.608793 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.609158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.609916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.612063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.629029 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:56.642921 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1269/2501] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:56.644828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.648751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1233/2502] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:56.652474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.715960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205025-df3a-4787-8c0c-b42063281bb0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.721120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19df0f-66cb-45b0-95a3-132acc3ee7c2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.721611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.722234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.722806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.722806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.723710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.725465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.726300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.726964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.727244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.727513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.728155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.730210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.749683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1270/2503] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:56.753795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.756762 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1234/2504] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:56.761196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.766225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3726b65-30a6-4390-a416-295bca561696 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.771201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1ca4-6e81-4717-8e29-d64d535ffd72 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.771858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.772674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.772953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.773224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.773935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.775500 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.776991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.777619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.777902 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.778171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.778829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.780420 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.788198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1271/2505] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:26:56.791002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1235/2506] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:26:56.792594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.796895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.807100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c486fe-e7f7-4cf1-b255-42ddf3d5b3e3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.807793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bf858-0282-460f-a403-5e48a7590836 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.811941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.812564 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.813047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.813316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.813986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.814567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.815662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.816405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.816980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.817266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.818087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.820627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.828531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1236/2507] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:26:56.832858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.844119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69343d88-dfc2-4bae-b6ab-7e66403d1fde admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.845592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1272/2508] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:26:56.850011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:56.850369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.851513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.851993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.852618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] PATH_INFO: `/v3/projects/c23917efa14f437fa209cf1542930786/users/71761b6bb94d40468244449ec2e06a1e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.868450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e743fb-944b-4ccf-b110-cfc6eb567013 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:56.874208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:56.875384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.875534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23917efa14f437fa209cf1542930786, user_id=71761b6bb94d40468244449ec2e06a1e, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.875728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.876024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] PATH_INFO: `/v3/projects/a7fe26cc9f2c4f54b6c5cd6c031bc82f/users/26f6493e102b43c898663625063a31cb/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.878319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b17a1b-d782-400b-b3dc-cc729b75cdf9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.895582 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7fe26cc9f2c4f54b6c5cd6c031bc82f, user_id=26f6493e102b43c898663625063a31cb, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:56.896218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1237/2509] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:56 2022] PUT /identity/v3/projects/c23917efa14f437fa209cf1542930786/users/71761b6bb94d40468244449ec2e06a1e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:56.903387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393c40f-d4d2-4816-8479-6a358e4394ac admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:56.907758 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.908091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.908308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.924437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1273/2510] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:56 2022] PUT /identity/v3/projects/a7fe26cc9f2c4f54b6c5cd6c031bc82f/users/26f6493e102b43c898663625063a31cb/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:26:56.935461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:56.935729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:56.935998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:56.946259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:56.957462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] MFA Rules not processed for user `71761b6bb94d40468244449ec2e06a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.960428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:56.969863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] MFA Rules not processed for user `26f6493e102b43c898663625063a31cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:56.971252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:57.066125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1238/2511] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:57.071610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1274/2512] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:57.075904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.078259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.079634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:57.122649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:57.147290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fdf5-130a-4066-ac90-b40c17deb4de None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.153261 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.153937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.154220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.154494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.155140 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.157138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.165750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc274b-bcc3-4c40-a543-f875977a7190 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.172483 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.173253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.173593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.174328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.189076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.192655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.200782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1239/2513] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:57 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:57.205139 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1275/2514] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:57.217698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.317112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fed0b-c9a7-43f4-b0fb-c6da982361c0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.321903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.322519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.322804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.323161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.323815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.326377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.362328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1240/2515] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:57 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:26:57.363554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.377850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f42e5d-b30b-48ea-874f-1affaa458bb2 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.383985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.384699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.385082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.385719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.386315 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.388701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.405762 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1276/2516] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:57.407457 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.423280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4f9b4-d925-4b75-91a3-866a0e32a556 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.429424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.430158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.430572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.430882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.431827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.433889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d028-cc50-4b8b-872b-5fadebb569c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.449521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1241/2517] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:26:57.454571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.475837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deaa-456e-4911-8e8d-e6d190dda843 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.488478 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.489192 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.489774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.489887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] PATH_INFO: `/v3/projects/71157bde6b2a4c5190b21e8a59a12b4f/users/29cb505070364125a4890a36f867a57c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.515481 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71157bde6b2a4c5190b21e8a59a12b4f, user_id=29cb505070364125a4890a36f867a57c, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.518468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.537972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1277/2518] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:57 2022] PUT /identity/v3/projects/71157bde6b2a4c5190b21e8a59a12b4f/users/29cb505070364125a4890a36f867a57c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:57.551298 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.551627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.552058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.574582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:57.603801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] MFA Rules not processed for user `29cb505070364125a4890a36f867a57c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:57.610435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:57.791849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1242/2519] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:26:57.799213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.901073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148483b-083d-4c03-b871-499cea889d86 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:57.909327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:57.909955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:57.910383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:57.910686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:57.911406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:57.913391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:57.956923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1278/2520] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:26:57 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:57.960110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:57.972947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b27232-e2db-4eee-b1d2-d3dbd8682d99 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.040273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.040957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.041307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.041645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.042395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.044409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.073893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1243/2521] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:26:57 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:26:58.075864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:58.087111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42f1d-2f20-49b5-9bba-9927b23b1e35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.091935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.092580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.092862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.093243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.093896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.095554 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.110847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1279/2522] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:58.112820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:58.125949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c5c6-5b8a-4b6c-91ff-b44571da98fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.130752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.131715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.132150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.132708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] PATH_INFO: `/v3/projects/cc0c59198381429ba659fc299ad10a55/users/234450fecfc44cc3acc1846acf3cc8fe/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.151207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0c59198381429ba659fc299ad10a55, user_id=234450fecfc44cc3acc1846acf3cc8fe, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.154506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.174636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1244/2523] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:58 2022] PUT /identity/v3/projects/cc0c59198381429ba659fc299ad10a55/users/234450fecfc44cc3acc1846acf3cc8fe/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:58.178867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:58.192331 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4fe48-a551-465b-8a8e-a447350a7cbc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.257140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:58.265977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.267152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.283621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.285574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.286551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.291738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4619789-7f2e-456c-a1d8-933ebe6b8239 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.310995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1280/2524] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:26:58.315319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:58.325874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ae7e-beb4-4409-a1a3-3998f3bf0d78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.332394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.333164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.333496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.333828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] PATH_INFO: `/v3/projects/cc0c59198381429ba659fc299ad10a55/users/234450fecfc44cc3acc1846acf3cc8fe/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.336160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0c59198381429ba659fc299ad10a55, user_id=234450fecfc44cc3acc1846acf3cc8fe, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.338887 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.356852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1245/2525] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:26:58 2022] PUT /identity/v3/projects/cc0c59198381429ba659fc299ad10a55/users/234450fecfc44cc3acc1846acf3cc8fe/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:26:58.361591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.362215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.362718 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.404665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] MFA Rules not processed for user `234450fecfc44cc3acc1846acf3cc8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:26:58.406596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:58.498576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1281/2526] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:26:58.510075 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:58.511949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:58.568049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:26:58.573713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0a95-5dfa-4489-9979-0330f00d120e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:58.578815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:58.579461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:58.579732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:58.580001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:58.593165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:58.593165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:58.602698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1246/2527] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:59.303395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:59.312712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89926-71e1-43cf-a38c-90fd41c6ac40 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:59.316312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:59.316968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:59.317246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:59.317518 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:59.393170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:59.394632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:59.411100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1282/2528] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:26:59.471577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:59.472461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:59.582882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce351c-3dcc-412b-ac4b-7d743f7d9ada service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:26:59.601965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:26:59.602487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:26:59.602688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:26:59.603123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] PATH_INFO: `/v3/users/5bb45c233ca243388bee05dede63415f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:26:59.634122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb45c233ca243388bee05dede63415f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:26:59.636673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:26:59.726193 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb45c233ca243388bee05dede63415f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:59.734555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb45c233ca243388bee05dede63415f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:59.735180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb45c233ca243388bee05dede63415f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:59.756082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb45c233ca243388bee05dede63415f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:59.756963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb45c233ca243388bee05dede63415f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:26:59.780737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1247/2529] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:26:59 2022] DELETE /identity/v3/users/5bb45c233ca243388bee05dede63415f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:26:59.971055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:59.972675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:26:59.977765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:26:59.979300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:00.037156 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbd3ea-31b5-4f89-b976-b2683cd96a1a service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.052094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:00.056013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a434ba-3af5-4466-bf07-681a64bc1b4a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.061631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.062531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.063032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.063397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] PATH_INFO: `/v3/projects/4a3dd0514b8542d280fbe2f798714ea3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.067720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:00.075204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3dd0514b8542d280fbe2f798714ea3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.077222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.088189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.088827 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.089141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.089412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.091377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:00.129695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3dd0514b8542d280fbe2f798714ea3', 'request_id': 'req-94758f93-32ca-426a-b499-e4ec9ac82702'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:00.136078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3dd0514b8542d280fbe2f798714ea3', 'request_id': 'req-94758f93-32ca-426a-b499-e4ec9ac82702'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:00.148748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3dd0514b8542d280fbe2f798714ea3', 'request_id': 'req-94758f93-32ca-426a-b499-e4ec9ac82702'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:00.157735 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.159173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65467c4b-af17-4ca2-af38-9f095bf4c063 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.162599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] The token cache is being invalidate because project 4a3dd0514b8542d280fbe2f798714ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:00.163128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:00.163746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-94758f93-32ca-426a-b499-e4ec9ac82702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:00.165495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1248/2530] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:26:59 2022] DELETE /identity/v3/projects/4a3dd0514b8542d280fbe2f798714ea3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:00.169520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1283/2531] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:00.170316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.170595 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.170860 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.190858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:00.192180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:00.253469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1249/2532] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:00.262170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.262441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.262711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.283059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:00.284322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:00.295625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1284/2533] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:00.299773 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.312335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf28af-9fad-440d-8b61-24a5295a7b12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.315805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.316449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.316831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.317102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.317760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:00.318061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.319665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.335109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1250/2534] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:00.336244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.345506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be379ad4-5ff6-4b93-abd6-6414e10ad46c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.350143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.350746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.351033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.351290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.351936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.353880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.382372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1285/2535] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:00.386969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.397427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3275dd-4bc6-4562-b8d2-0ba25f9c752f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.454852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.455459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.455827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.456078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.456883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.458799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.471267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:00.487615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1251/2536] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:00.488959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.499879 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336667a-306a-44ca-9485-73366d12955b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.505299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.505914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.506194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.506499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.507111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.508877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.524553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1286/2537] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:00.526176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.536863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f816-b205-43ca-a5df-e48f1f9e5762 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.542233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.543089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.543430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.543704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.544421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.546438 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe44994-6898-4cdf-8dc2-5ba04620ddac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.558723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1252/2538] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:00.562986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.573232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e032-5893-47a4-b600-1ed851af01c5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.577877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.578543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.578821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.579136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] PATH_INFO: `/v3/projects/81931c0712ff4469a6b97f717997590b/users/f92fd6cea4ef4eddaccc2ac6e4081dc0/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.595212 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81931c0712ff4469a6b97f717997590b, user_id=f92fd6cea4ef4eddaccc2ac6e4081dc0, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.598034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.612555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1287/2539] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:00 2022] PUT /identity/v3/projects/81931c0712ff4469a6b97f717997590b/users/f92fd6cea4ef4eddaccc2ac6e4081dc0/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:00.624727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.625182 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.625303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.655573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] MFA Rules not processed for user `f92fd6cea4ef4eddaccc2ac6e4081dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:00.656842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:00.726812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1253/2540] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:00.731117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.795039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6660-d44b-4905-af92-668065e1c538 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.800020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.800727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.800999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.801266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.801890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.803789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.832437 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1288/2541] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:00.836669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.850142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df1373-1006-4ab2-b5fb-5aec6a50cacb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.914078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.914607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.914885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.915155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.915799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.917750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.941109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1254/2542] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:00 2022] POST /identity/v3/users => generated 307 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:00.945076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.950662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:00.955773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bdc90-9ffe-4a44-9831-debeb5d2837a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.962314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.962950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.963230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.963496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:00.964161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:00.965784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:00.977173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1289/2543] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:00.981456 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:00.992221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658108-7941-4625-b546-596ecc4c9a4c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:00.997035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:00.997759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:00.998100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:00.998440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] PATH_INFO: `/v3/projects/76d2b7d657b349438beed450ba1fa078/users/7e85d90f98b648bf86a4c05525df9362/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:01.013695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d2b7d657b349438beed450ba1fa078, user_id=7e85d90f98b648bf86a4c05525df9362, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:01.016367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:01.037133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1255/2544] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:00 2022] PUT /identity/v3/projects/76d2b7d657b349438beed450ba1fa078/users/7e85d90f98b648bf86a4c05525df9362/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:01.041164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:01.052342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363077f8-774d-4d05-a66e-ffe54c8456e6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:01.109084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:01.109664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:01.109936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:01.110241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:01.110888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:01.112475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379ad74-77b1-49df-aced-a27895ece276 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:01.122605 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1290/2545] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:01.126628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:01.137001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eea6c1-71dc-47e3-bf7d-bc39e731d716 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:01.142022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:01.142672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:01.142947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:01.143234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] PATH_INFO: `/v3/projects/76d2b7d657b349438beed450ba1fa078/users/7e85d90f98b648bf86a4c05525df9362/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:01.145145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d2b7d657b349438beed450ba1fa078, user_id=7e85d90f98b648bf86a4c05525df9362, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:01.147744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:01.158224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1256/2546] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:01 2022] PUT /identity/v3/projects/76d2b7d657b349438beed450ba1fa078/users/7e85d90f98b648bf86a4c05525df9362/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:01.168648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:01.168870 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:01.169197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:01.199920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] MFA Rules not processed for user `7e85d90f98b648bf86a4c05525df9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:01.201161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:01.272359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1291/2547] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:01.284278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:01.286166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:01.319109 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:01.351145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d4f3-44fe-46c2-ab20-709bec33f568 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:01.359688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:01.360423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:01.360795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:01.361053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:01.371534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:01.373035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf446-422f-4789-a465-b8ff0de50294 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:01.383002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1257/2548] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:02.922158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:02.994693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a632b4-86a1-44be-9277-2810d3671c08 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:03.007543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:03.008328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:03.008739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:03.009123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] PATH_INFO: `/v3/users/f92fd6cea4ef4eddaccc2ac6e4081dc0` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:03.010374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92fd6cea4ef4eddaccc2ac6e4081dc0)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:03.012626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:03.028357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:03.064562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92fd6cea4ef4eddaccc2ac6e4081dc0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.071841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92fd6cea4ef4eddaccc2ac6e4081dc0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.085097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92fd6cea4ef4eddaccc2ac6e4081dc0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.094614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92fd6cea4ef4eddaccc2ac6e4081dc0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.095253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92fd6cea4ef4eddaccc2ac6e4081dc0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.101592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1292/2549] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:02 2022] DELETE /identity/v3/users/f92fd6cea4ef4eddaccc2ac6e4081dc0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:03.106222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:03.106705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:03.106820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:03.126477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:03.127717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:03.193973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1258/2550] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:03.203896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:03.205461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:03.267781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad16fb7-a8d5-426c-ba3d-674d798ad95b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:03.271737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:03.272350 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:03.272666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:03.272939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:03.282258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:03.283703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:03.293455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1293/2551] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:03.615642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:03.617537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:03.639672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d3c27-dfca-42e4-9016-dbca3668efa5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:03.649462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:03.650207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:03.650580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:03.650783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] PATH_INFO: `/v3/projects/81931c0712ff4469a6b97f717997590b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:03.651940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81931c0712ff4469a6b97f717997590b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:03.654013 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:03.706156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81931c0712ff4469a6b97f717997590b', 'request_id': 'req-da2c3c1b-93ac-4508-9f12-0f569dd73a83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.712300 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81931c0712ff4469a6b97f717997590b', 'request_id': 'req-da2c3c1b-93ac-4508-9f12-0f569dd73a83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.719371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81931c0712ff4469a6b97f717997590b', 'request_id': 'req-da2c3c1b-93ac-4508-9f12-0f569dd73a83'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.732993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] The token cache is being invalidate because project 81931c0712ff4469a6b97f717997590b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:03.733291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.733947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.735553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1259/2552] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:03 2022] DELETE /identity/v3/projects/81931c0712ff4469a6b97f717997590b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:03.739654 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:03.749782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b02d-de2a-4234-98cd-017322d0ef6f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:03.810461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:03.811100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:03.811388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:03.811664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] PATH_INFO: `/v3/users/7e85d90f98b648bf86a4c05525df9362` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:03.812819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e85d90f98b648bf86a4c05525df9362)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:03.814836 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:03.857639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:03.868903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e85d90f98b648bf86a4c05525df9362'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.876843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e85d90f98b648bf86a4c05525df9362'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.889014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e85d90f98b648bf86a4c05525df9362'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.898152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e85d90f98b648bf86a4c05525df9362'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.898788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d5f79ead-8f69-40e5-b93a-b53172b086ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e85d90f98b648bf86a4c05525df9362'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:03.904903 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1294/2553] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:03 2022] DELETE /identity/v3/users/7e85d90f98b648bf86a4c05525df9362 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:04.184844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.186795 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:04.202104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c3c1b-93ac-4508-9f12-0f569dd73a83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.256358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:04.266569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.267402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.268021 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.268402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] PATH_INFO: `/v3/projects/76d2b7d657b349438beed450ba1fa078` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.269801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d2b7d657b349438beed450ba1fa078)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.271996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.318713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d2b7d657b349438beed450ba1fa078', 'request_id': 'req-e17a12a6-b01a-4e50-9833-f64d14e04a06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:04.324462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d2b7d657b349438beed450ba1fa078', 'request_id': 'req-e17a12a6-b01a-4e50-9833-f64d14e04a06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:04.330688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d2b7d657b349438beed450ba1fa078', 'request_id': 'req-e17a12a6-b01a-4e50-9833-f64d14e04a06'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:04.344186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] The token cache is being invalidate because project 76d2b7d657b349438beed450ba1fa078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:04.344562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:04.345147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e17a12a6-b01a-4e50-9833-f64d14e04a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:04.346625 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1260/2554] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:04 2022] DELETE /identity/v3/projects/76d2b7d657b349438beed450ba1fa078 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:04.351523 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.352050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.352050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.372051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:04.373271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:04.461954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1295/2555] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:04.470110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.470387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.470662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.491982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:04.493293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:04.502673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1261/2556] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:04.507425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.517721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220fa0d-950f-44f7-9a7e-d148d7659ae6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.523031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.523618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.523895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.524165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.524859 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:04.525160 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.526769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.539443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1296/2557] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:04.543867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.554535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500c239-42ee-4007-8f5d-b101b8641cff None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.560283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.560920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.561209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.561473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.562112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.564108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.594620 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1262/2558] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:04.599052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.609645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97029eb4-4e75-46c8-bf3b-af96eea8ee83 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.668519 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.669265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.669543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.670157 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.670784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.672749 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.696542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1297/2559] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:04.701122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.712465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b236778-5525-42ee-9130-1e6c579e77d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.717725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.718430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.718780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.719176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.720027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.721847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.736441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1263/2560] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:04.738028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.749017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb647b-f6ad-4b9a-9a38-6b8023602b56 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.753898 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.754504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.754780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.755051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.755698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.757359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60624880-990c-4f80-8fa5-82598e67bc7b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.768675 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1298/2561] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:04.773042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:04.783264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8351b2-3ff7-471e-9d79-ed428395d51e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:04.788305 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:04.789060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.789405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.789741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] PATH_INFO: `/v3/projects/8faa3025eeb14ee586b0cfff2327c730/users/d74eb78bae394bebb70d159b24f7c98e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.805739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8faa3025eeb14ee586b0cfff2327c730, user_id=d74eb78bae394bebb70d159b24f7c98e, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:04.808441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:04.823311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1264/2562] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:04 2022] PUT /identity/v3/projects/8faa3025eeb14ee586b0cfff2327c730/users/d74eb78bae394bebb70d159b24f7c98e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:04.835095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:04.835377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:04.835645 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:04.864529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] MFA Rules not processed for user `d74eb78bae394bebb70d159b24f7c98e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:04.865770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:04.874046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:04.934216 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1299/2563] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:04.938989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.003199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ef86d-e0b8-4d2a-97cd-9e8986558c36 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.008548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.009157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.009445 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.009759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.010542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.012697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.042642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1265/2564] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:04 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:05.047370 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.058610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82736abd-28d3-4a48-b2dd-a5eb704a7d16 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.118912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.119531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.119805 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.120091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.120768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.122685 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.145354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1300/2565] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:05 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:05.151487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.160589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:05.164805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234a15d-b06d-4c61-b86d-96fcc36c1092 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.171594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.172209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.172631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.173133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.173987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.175811 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.186757 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1266/2566] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:05.191396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.202362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26357-339d-4aa4-a9a0-e54ba8f2a522 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.208059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.208782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.209068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.209340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] PATH_INFO: `/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5/users/61611384d919426b90fe81a52700363b/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.225225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1a920bf2bc461a9ef779ff2b1227b5, user_id=61611384d919426b90fe81a52700363b, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.227891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.243916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1301/2567] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:05 2022] PUT /identity/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5/users/61611384d919426b90fe81a52700363b/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:05.248047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.258798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fde57-6ad0-4541-9cd2-719e71e4ab6a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.315432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.316054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.316387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.316692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.317343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.318931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c9ca-e787-4953-95ea-ae7adb982f97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.331580 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1267/2568] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:05.332800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.343028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33194fe0-654b-4b0c-b783-9bcb15729e7c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.347670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.348327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.348642 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.348916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] PATH_INFO: `/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5/users/61611384d919426b90fe81a52700363b/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.350901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1a920bf2bc461a9ef779ff2b1227b5, user_id=61611384d919426b90fe81a52700363b, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.353604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.364069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1302/2569] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:05 2022] PUT /identity/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5/users/61611384d919426b90fe81a52700363b/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:05.374477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.374761 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.375033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.403562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] MFA Rules not processed for user `61611384d919426b90fe81a52700363b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:05.404906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:05.480026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1268/2570] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:05.492028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.493963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:05.563799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afbf6-5957-4f1b-9fd7-0f0a26120175 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.567899 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.568542 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.568824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.569096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.582545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.584036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91555938-b1c9-4cd7-a57c-78b52dc08d19 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.594623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1303/2571] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:05.657661 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.723326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e6c89-d002-438a-b3fa-d316275afe36 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.729181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.729836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.730119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.730375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] PATH_INFO: `/v3/users/d74eb78bae394bebb70d159b24f7c98e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.731562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74eb78bae394bebb70d159b24f7c98e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.733548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:05.750288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:05.783177 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74eb78bae394bebb70d159b24f7c98e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:05.790036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74eb78bae394bebb70d159b24f7c98e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:05.790770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74eb78bae394bebb70d159b24f7c98e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:05.802422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74eb78bae394bebb70d159b24f7c98e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:05.803190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74eb78bae394bebb70d159b24f7c98e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:05.822125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1269/2572] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:05 2022] DELETE /identity/v3/users/d74eb78bae394bebb70d159b24f7c98e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:05.826796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.827079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.827346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.837659 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:05.858680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:05.859960 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:05.931241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1304/2573] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:05.938378 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.939706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:05.939949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:05.941259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:05.952872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f8f0-afd6-4802-984a-83691155b24f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:05.957540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:05.958166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:05.958441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:05.958712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] PATH_INFO: `/v3/users/71761b6bb94d40468244449ec2e06a1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:05.959845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=71761b6bb94d40468244449ec2e06a1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:05.961814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.004106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17621e5-d498-4241-8409-14090e331aa9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.008048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.008854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.009302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.009723 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.015704 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71761b6bb94d40468244449ec2e06a1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.020287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.021844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e014b4b-11b8-4e97-a932-3e81cb0a7fe8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.024942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71761b6bb94d40468244449ec2e06a1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.025707 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71761b6bb94d40468244449ec2e06a1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.032195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1305/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:06.034873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71761b6bb94d40468244449ec2e06a1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.035801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71761b6bb94d40468244449ec2e06a1e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.057146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1270/2575] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:05 2022] DELETE /identity/v3/users/71761b6bb94d40468244449ec2e06a1e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:06.061725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.062078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.062464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.083608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:06.084917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.145426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1306/2576] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:06.156041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.157939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.224544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55954b-466b-4a91-909e-76c95654ef1d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.228394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.229067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.229319 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.229582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.240118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.241582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.252892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1271/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:06.353724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.355198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.368487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa2f5-1c74-49b5-a38a-de20134176b4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.374116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.374767 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.375205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.375662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] PATH_INFO: `/v3/projects/8faa3025eeb14ee586b0cfff2327c730` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.376954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8faa3025eeb14ee586b0cfff2327c730)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.379302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.431688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:06.435346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8faa3025eeb14ee586b0cfff2327c730', 'request_id': 'req-454c0582-02df-4d64-91ea-c1c7c078c4c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.445992 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8faa3025eeb14ee586b0cfff2327c730', 'request_id': 'req-454c0582-02df-4d64-91ea-c1c7c078c4c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.452671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8faa3025eeb14ee586b0cfff2327c730', 'request_id': 'req-454c0582-02df-4d64-91ea-c1c7c078c4c6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.466657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] The token cache is being invalidate because project 8faa3025eeb14ee586b0cfff2327c730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:06.466924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.467665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.469342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1307/2578] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:06 2022] DELETE /identity/v3/projects/8faa3025eeb14ee586b0cfff2327c730 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:06.473544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.483265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c9a32-4d7d-4850-91d3-dc39da88d8a6 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.487996 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:06.543387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.544030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.544309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.545008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] PATH_INFO: `/v3/users/61611384d919426b90fe81a52700363b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.546096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=61611384d919426b90fe81a52700363b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.548283 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.601632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.604116 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.622634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0582-02df-4d64-91ea-c1c7c078c4c6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.624487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61611384d919426b90fe81a52700363b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.628776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.629680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.629965 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.630323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] PATH_INFO: `/v3/projects/c23917efa14f437fa209cf1542930786` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.631511 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23917efa14f437fa209cf1542930786)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.633415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61611384d919426b90fe81a52700363b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.633562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.635082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61611384d919426b90fe81a52700363b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.647123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61611384d919426b90fe81a52700363b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.648119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a5b78c64-334b-49ad-a0c5-eb1b16745ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61611384d919426b90fe81a52700363b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.669359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1272/2579] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:06 2022] DELETE /identity/v3/users/61611384d919426b90fe81a52700363b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:06.697443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23917efa14f437fa209cf1542930786', 'request_id': 'req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.708406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23917efa14f437fa209cf1542930786', 'request_id': 'req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.717933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23917efa14f437fa209cf1542930786', 'request_id': 'req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.731794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] The token cache is being invalidate because project c23917efa14f437fa209cf1542930786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:06.732184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.732953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:06.734809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1308/2580] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:06 2022] DELETE /identity/v3/projects/c23917efa14f437fa209cf1542930786 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:06.739404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98d7a-cbe9-4f57-8315-4b512c0773a3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.739681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98d7a-cbe9-4f57-8315-4b512c0773a3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.739950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98d7a-cbe9-4f57-8315-4b512c0773a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.762392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ddf98d7a-cbe9-4f57-8315-4b512c0773a3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:06.763523 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ddf98d7a-cbe9-4f57-8315-4b512c0773a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.813301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.814803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.840395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1273/2581] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:06.842012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.842343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.842566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.864842 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:06.866615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.875481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1274/2582] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:06.879365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.890094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4b06f-d3a1-4b26-9dea-d9cbcef38e92 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.890510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797e81-95cc-43ec-9d5b-f398b5e38ae9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.894594 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.895312 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.895689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.895789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.896151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.896814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.896814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.897027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.897637 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:06.897900 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.898485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.899534 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.912645 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1275/2583] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:06.916804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.927668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5525a-a25a-44fc-bacb-cf6cb054864f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:06.932296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:06.933038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:06.933314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:06.934316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:06.934316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.936073 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.942183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:06.966835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:06.968568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:06.970944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1276/2584] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:06 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:06.972162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.979721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1309/2585] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:06.980595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:06.981961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:06.997910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5820-a634-4074-93da-3368aae47347 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.003533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:07.060854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da29094-0f25-4bf9-bb03-513bd2e8aa4c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.065163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.065803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.066081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.066357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] PATH_INFO: `/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.067479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1a920bf2bc461a9ef779ff2b1227b5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.068003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.069367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.069598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.069999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.070425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.071221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.076184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.104540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1277/2586] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:06 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:07.108067 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.119347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c2fd-7fe1-411d-9edb-72250dd1c203 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.126050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1a920bf2bc461a9ef779ff2b1227b5', 'request_id': 'req-b9822e84-2daf-400f-afa1-f1ca48459015'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:07.134946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1a920bf2bc461a9ef779ff2b1227b5', 'request_id': 'req-b9822e84-2daf-400f-afa1-f1ca48459015'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:07.143383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1a920bf2bc461a9ef779ff2b1227b5', 'request_id': 'req-b9822e84-2daf-400f-afa1-f1ca48459015'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:07.155929 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] The token cache is being invalidate because project ee1a920bf2bc461a9ef779ff2b1227b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:07.156225 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:07.156948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b9822e84-2daf-400f-afa1-f1ca48459015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:07.159035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1310/2587] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:06 2022] DELETE /identity/v3/projects/ee1a920bf2bc461a9ef779ff2b1227b5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:07.164119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d70-ac75-4abf-8560-7e51fcc0b6ff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.164401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d70-ac75-4abf-8560-7e51fcc0b6ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.164702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d70-ac75-4abf-8560-7e51fcc0b6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.185218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.185859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.186010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-032d7d70-ac75-4abf-8560-7e51fcc0b6ff None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:07.186121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.186809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.187244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-032d7d70-ac75-4abf-8560-7e51fcc0b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.187450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.189541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.198507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1311/2588] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:07.200755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1278/2589] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:07.207011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.207502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.207778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.208789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.211020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.228061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06672f29-03d9-4748-a710-294570023b9d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.228729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:07.229692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.233442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.234058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.234342 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.234615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.235429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.237036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.239327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1312/2590] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:07.243789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.248751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1279/2591] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:07.253277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.256421 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd260-143d-4f4f-afdd-41294742db1d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.261906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.262524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.263004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.263224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.263875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:07.264268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.265819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.269069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df259a-f875-4a11-a019-45afdbfe5918 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.275964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.276720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.277003 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.277282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] PATH_INFO: `/v3/projects/6157595f6bd44f4589aef318fd7efc77/users/8fbed53b22ce4a11a3eda1f3d4f7e710/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.278666 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1313/2592] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:07.283220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.295626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b008-b502-42f3-ad7f-5b701198516a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.296392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6157595f6bd44f4589aef318fd7efc77, user_id=8fbed53b22ce4a11a3eda1f3d4f7e710, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.299990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cbaf7-bad8-499a-89a3-d9ea6bf229a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.301912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.302671 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.302949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.303557 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.304368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.306636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.315401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1280/2593] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:07 2022] PUT /identity/v3/projects/6157595f6bd44f4589aef318fd7efc77/users/8fbed53b22ce4a11a3eda1f3d4f7e710/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:07.325748 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.326036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.326306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.339820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1314/2594] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:07.344014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.357209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] MFA Rules not processed for user `8fbed53b22ce4a11a3eda1f3d4f7e710`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:07.358561 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.390716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:07.400092 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:07.414619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6f68-a215-40ef-928d-7582daae040c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.421224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.422121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.422699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.422969 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.423608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.425915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.438171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1281/2595] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:07.439208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.450385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1315/2596] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:07.450834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17b6e3-fd11-46ea-a6e7-65e0fefb7419 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.454825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.456604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.457676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.457956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.458362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.459001 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.461237 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.466948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4992-ffb4-467e-9b23-de1312dfb3e6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.472161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.472803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.473088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.473358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.474009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.475611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.488931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1316/2597] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:07.490869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.493548 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1282/2598] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:07.497348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.501503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da89c-6cd6-4ee8-8605-6a0f0d97a936 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.563607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.564230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.564522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.564944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.565598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.566217 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d83ed-f42e-4ca8-92af-5e5e3284e5b1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.567301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.571179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.571895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.572176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.572802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.573106 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.575005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.578555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1317/2599] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:07.582223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.595489 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123a842-37fe-4bd1-b001-1c71d70075fb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.601493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.602491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.603015 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.603469 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] PATH_INFO: `/v3/projects/d47f2bfdb11549c3b09b7bd20c3722a3/users/782b5055962b46d69e447a95a08ec972/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.606906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1283/2600] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:07.617695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.626153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47f2bfdb11549c3b09b7bd20c3722a3, user_id=782b5055962b46d69e447a95a08ec972, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.628026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f960c4-a308-465f-b5ad-5f5d34fbe693 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.628820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.634007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.634805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.635142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.635476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.636220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.638099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86927a68-fb19-432c-b57b-dbc7616d4eab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.654191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1318/2601] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:07 2022] PUT /identity/v3/projects/d47f2bfdb11549c3b09b7bd20c3722a3/users/782b5055962b46d69e447a95a08ec972/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:07.654422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1284/2602] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:07.654521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.655785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.656156 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.656536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.688065 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] MFA Rules not processed for user `782b5055962b46d69e447a95a08ec972`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:07.689370 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.727887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f1de3-f22e-4493-96ea-598445903e12 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.732911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.733678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.733973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.734180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] PATH_INFO: `/v3/projects/4b83359c891849a3bd8a37734fce48c3/users/d60c44a7b119424e8b473176374c0235/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.752608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b83359c891849a3bd8a37734fce48c3, user_id=d60c44a7b119424e8b473176374c0235, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.755923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.767020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1285/2603] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:07.771484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.774710 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1319/2604] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:07 2022] PUT /identity/v3/projects/4b83359c891849a3bd8a37734fce48c3/users/d60c44a7b119424e8b473176374c0235/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:07.779442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.788477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ffd47-5e27-4e7d-833f-9c00b302fc95 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.802817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:07.809487 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:07.846602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd35353-3d69-4cca-b31c-9e4831901a47 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.851327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.851958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.852233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.852493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.852895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.853158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.853510 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.853790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.854058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.854725 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.854829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.856666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.868327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1320/2605] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:07.869458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.879106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e029-599e-4c97-ae2e-7ebf6b7d1f63 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:07.883712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:07.884342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.884647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.884916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] PATH_INFO: `/v3/projects/4b83359c891849a3bd8a37734fce48c3/users/d60c44a7b119424e8b473176374c0235/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.885329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1286/2606] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:07.886964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b83359c891849a3bd8a37734fce48c3, user_id=d60c44a7b119424e8b473176374c0235, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:07.889226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:07.889744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a7aa2-9d75-4f93-a3ce-4d8980171d6e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:07.900060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1321/2607] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:07 2022] PUT /identity/v3/projects/4b83359c891849a3bd8a37734fce48c3/users/d60c44a7b119424e8b473176374c0235/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:07.911914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:07.912193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:07.912458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:07.940202 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] MFA Rules not processed for user `d60c44a7b119424e8b473176374c0235`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:07.941562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:07.950884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d5d3-13f8-42e4-a8ab-27a9afc3178e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.008255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.008983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.009270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.009612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.010288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.012491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.014753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1322/2608] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:08.026307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.027809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:08.045674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1287/2609] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:07 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:08.046853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.056841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a7b9e-5974-4152-9adc-9d9d1721aceb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.061511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.062107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.063026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.063312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.064277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.067010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.081486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1288/2610] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:08.082731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.087643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedac059-bfe4-4e05-942b-df7b0de222ed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.091296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.091911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.092186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.092455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.093186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140df3f-e3c5-4ec6-b18c-413dddc42f1c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.097864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.098524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.098807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.099085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] PATH_INFO: `/v3/projects/01c45a02bb234f039499b38486bb49a5/users/31560faf96234981b143f81637610a94/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.103543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.105032 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.113994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c45a02bb234f039499b38486bb49a5, user_id=31560faf96234981b143f81637610a94, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.114689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1323/2611] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:08.116671 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.128924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:08.134130 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1289/2612] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:08 2022] PUT /identity/v3/projects/01c45a02bb234f039499b38486bb49a5/users/31560faf96234981b143f81637610a94/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:08.138461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.154799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b35b-09d0-4957-b1b5-086ca67f9840 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.189551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:08.218011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.218769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.219082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.219486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.220236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.221994 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1fb5-6760-4156-ba5d-b1cd5403ba8e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.232306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1324/2613] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:08.237027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.250184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6582d0-95ad-46a3-a5e0-511c0adab0dd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.255865 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.256557 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.256835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.257102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] PATH_INFO: `/v3/projects/01c45a02bb234f039499b38486bb49a5/users/31560faf96234981b143f81637610a94/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.259138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c45a02bb234f039499b38486bb49a5, user_id=31560faf96234981b143f81637610a94, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.262005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.278147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1290/2614] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:08 2022] PUT /identity/v3/projects/01c45a02bb234f039499b38486bb49a5/users/31560faf96234981b143f81637610a94/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:08.284734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577501-aafe-4f17-83b0-14e22660103f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.284964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577501-aafe-4f17-83b0-14e22660103f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.285242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577501-aafe-4f17-83b0-14e22660103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.318099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-22577501-aafe-4f17-83b0-14e22660103f None None] MFA Rules not processed for user `31560faf96234981b143f81637610a94`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:08.319345 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-22577501-aafe-4f17-83b0-14e22660103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:08.392908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1325/2615] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:08.403463 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.462912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88d84d-800b-4caa-8605-a07ff6f40bd4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.466951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.467573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.467847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.468115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.477778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.479285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.489426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1291/2616] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:08.521237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22577501-aafe-4f17-83b0-14e22660103f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.538018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.539466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:08.550981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd51b06-72a9-43fa-8576-dec427c6d64c service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.554592 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.555183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.555458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.555728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.589911 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22577501-aafe-4f17-83b0-14e22660103f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.593943 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.594628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.594901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.595165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] PATH_INFO: `/v3/users/8fbed53b22ce4a11a3eda1f3d4f7e710` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.596291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fbed53b22ce4a11a3eda1f3d4f7e710)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.598252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.614361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.615820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e57ce-3344-4923-b74d-4b5e4f3ea45d service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.625189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1292/2617] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:08.647496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fbed53b22ce4a11a3eda1f3d4f7e710'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:08.654753 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fbed53b22ce4a11a3eda1f3d4f7e710'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:08.656655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:08.667852 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fbed53b22ce4a11a3eda1f3d4f7e710'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:08.676700 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fbed53b22ce4a11a3eda1f3d4f7e710'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:08.677307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fbed53b22ce4a11a3eda1f3d4f7e710'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:08.683580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1326/2618] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:08 2022] DELETE /identity/v3/users/8fbed53b22ce4a11a3eda1f3d4f7e710 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:08.687948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.688270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.688591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.707490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:08.708736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:08.772103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1293/2619] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:08.782286 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:08.784096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:08.854079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd068-7a86-4aa7-b9f9-41b895e6ec9c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:08.857792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:08.858411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:08.858692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:08.858968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:08.870968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:08.873905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:08.894033 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1327/2620] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:09.171339 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:09.173180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.195504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071eb94-7e31-4f1d-b891-fd6a5f794db7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:09.201990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:09.202806 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.203120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.203464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] PATH_INFO: `/v3/projects/6157595f6bd44f4589aef318fd7efc77` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.204680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6157595f6bd44f4589aef318fd7efc77)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:09.206936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:09.237629 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:09.265841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6157595f6bd44f4589aef318fd7efc77', 'request_id': 'req-8df970e1-8287-4697-a54b-5b4d50e8baa3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.272070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6157595f6bd44f4589aef318fd7efc77', 'request_id': 'req-8df970e1-8287-4697-a54b-5b4d50e8baa3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.278616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6157595f6bd44f4589aef318fd7efc77', 'request_id': 'req-8df970e1-8287-4697-a54b-5b4d50e8baa3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.286359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:09.287874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.293417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] The token cache is being invalidate because project 6157595f6bd44f4589aef318fd7efc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:09.293709 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.294328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.296203 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1294/2621] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:09 2022] DELETE /identity/v3/projects/6157595f6bd44f4589aef318fd7efc77 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:09.300280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:09.301242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419d3bc-f61e-42b4-abfa-df22db34fea2 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:09.301606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.356604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:09.357141 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.357411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.357677 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] PATH_INFO: `/v3/users/26f6493e102b43c898663625063a31cb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.358838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f6493e102b43c898663625063a31cb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:09.360851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:09.364367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df970e1-8287-4697-a54b-5b4d50e8baa3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:09.370417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:09.371055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.371338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.371610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] PATH_INFO: `/v3/users/d60c44a7b119424e8b473176374c0235` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.372809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60c44a7b119424e8b473176374c0235)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:09.374727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:09.409461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f6493e102b43c898663625063a31cb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.416838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f6493e102b43c898663625063a31cb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.423031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60c44a7b119424e8b473176374c0235'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.429857 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f6493e102b43c898663625063a31cb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.430116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60c44a7b119424e8b473176374c0235'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.430827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60c44a7b119424e8b473176374c0235'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.438254 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f6493e102b43c898663625063a31cb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.439000 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dd89ff04-e19d-4160-880e-8743af9d6dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f6493e102b43c898663625063a31cb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.440669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:09.441443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60c44a7b119424e8b473176374c0235'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.442321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60c44a7b119424e8b473176374c0235'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.446798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1328/2622] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:09 2022] DELETE /identity/v3/users/26f6493e102b43c898663625063a31cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:09.451649 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.452362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.452918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.461839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1295/2623] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:09 2022] DELETE /identity/v3/users/d60c44a7b119424e8b473176374c0235 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:09.473035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:09.474270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.537287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1329/2624] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:09.547656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:09.549100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.587433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:09.613027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400477-ad96-41cc-9eb8-bc6545ebb587 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:09.616624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:09.617231 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.617504 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.617775 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.627486 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:09.628974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0a92-cb52-423b-b3f0-5482aa6d9a10 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:09.639062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1296/2625] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:09.826004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:09.827602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:09.840556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618096c-71d0-4725-95fb-7c5f80db3cd1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:09.846364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:09.847146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.847424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.847690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] PATH_INFO: `/v3/projects/4b83359c891849a3bd8a37734fce48c3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.848949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b83359c891849a3bd8a37734fce48c3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:09.850865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:09.920255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b83359c891849a3bd8a37734fce48c3', 'request_id': 'req-456696b8-d0c7-40bb-a45f-c3f985c100b7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.929274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b83359c891849a3bd8a37734fce48c3', 'request_id': 'req-456696b8-d0c7-40bb-a45f-c3f985c100b7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.936546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b83359c891849a3bd8a37734fce48c3', 'request_id': 'req-456696b8-d0c7-40bb-a45f-c3f985c100b7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.946887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] The token cache is being invalidate because project 4b83359c891849a3bd8a37734fce48c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:09.947175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.947810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:09.949570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1330/2626] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:09 2022] DELETE /identity/v3/projects/4b83359c891849a3bd8a37734fce48c3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:09.954673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c585bc-9cd7-4018-829d-0c2310e1dcd5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:09.954962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c585bc-9cd7-4018-829d-0c2310e1dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:09.955242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c585bc-9cd7-4018-829d-0c2310e1dcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:09.976495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-74c585bc-9cd7-4018-829d-0c2310e1dcd5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:09.977648 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74c585bc-9cd7-4018-829d-0c2310e1dcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.006793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.021847 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.073574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1297/2627] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:10.081679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.082045 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.082380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.097411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456696b8-d0c7-40bb-a45f-c3f985c100b7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.102884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:10.103038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.103792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.104080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.104331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.104463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] PATH_INFO: `/v3/projects/a7fe26cc9f2c4f54b6c5cd6c031bc82f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.106287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7fe26cc9f2c4f54b6c5cd6c031bc82f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.108282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.116069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1298/2628] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:10.117569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.127483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ca132-79b7-448d-95ca-ec779c3a85ac None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.132432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.133072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.133354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.133631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.134292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:10.134598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.136207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:10.136405 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.152249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1299/2629] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:10.156489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.166547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db4285-3a45-4fa6-90e9-00d7d81c0a9a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.168090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7fe26cc9f2c4f54b6c5cd6c031bc82f', 'request_id': 'req-8e205ba1-2257-4361-a3dc-1273e985ddd7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.178205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7fe26cc9f2c4f54b6c5cd6c031bc82f', 'request_id': 'req-8e205ba1-2257-4361-a3dc-1273e985ddd7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.186618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7fe26cc9f2c4f54b6c5cd6c031bc82f', 'request_id': 'req-8e205ba1-2257-4361-a3dc-1273e985ddd7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.198631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] The token cache is being invalidate because project a7fe26cc9f2c4f54b6c5cd6c031bc82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:10.198958 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.199760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.201392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1331/2630] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:10 2022] DELETE /identity/v3/projects/a7fe26cc9f2c4f54b6c5cd6c031bc82f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:10.208630 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.210031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.239508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.241053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.241596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.241867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.242522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.244446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.274190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1300/2631] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:10.278425 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.279994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.281572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e205ba1-2257-4361-a3dc-1273e985ddd7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.326162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:10.348472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.349474 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.349944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.350395 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] PATH_INFO: `/v3/users/29cb505070364125a4890a36f867a57c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.351238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16679987-4ee1-4931-81e2-fc7c6bbce267 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.352146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29cb505070364125a4890a36f867a57c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.354838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.355952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.356594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.356876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.357148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.357784 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.359690 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.384613 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1301/2632] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:10.387293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.398406 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53fde8-ab05-437d-b83c-434e3e07eb17 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.403642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.404368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.404737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.405070 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.405776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.407494 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.418786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1302/2633] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:10.420789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29cb505070364125a4890a36f867a57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.423146 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.428178 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29cb505070364125a4890a36f867a57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.432727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:10.433720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac645-c78b-4587-88ef-8c7cc9f6aeef admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.438914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.439520 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.439816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.440086 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.440768 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.442208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29cb505070364125a4890a36f867a57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.442348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.453380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29cb505070364125a4890a36f867a57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.453967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1303/2634] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:10.454401 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a239648c-a0e4-4f78-9a40-03408d6f471f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29cb505070364125a4890a36f867a57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:10.457946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.459987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.464349 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1332/2635] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:10 2022] DELETE /identity/v3/users/29cb505070364125a4890a36f867a57c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:10.474008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5cab-96d6-4007-b248-0db782deefb1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.550213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.550891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.551204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.551454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] PATH_INFO: `/v3/projects/935545f8c1ca424197ce9c86aa23c616/users/fcfe348dd5e44f68b73c4d2a5b37404a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.569569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=935545f8c1ca424197ce9c86aa23c616, user_id=fcfe348dd5e44f68b73c4d2a5b37404a, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.572215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.604733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1304/2636] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:10 2022] PUT /identity/v3/projects/935545f8c1ca424197ce9c86aa23c616/users/fcfe348dd5e44f68b73c4d2a5b37404a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:10.608626 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.609131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.609613 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.653694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] MFA Rules not processed for user `fcfe348dd5e44f68b73c4d2a5b37404a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:10.654997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.738208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1333/2637] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:10.739185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.790394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.795017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:10.817997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c507c81-ace5-4952-bf7f-042287eed72d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.822760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.823682 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.823914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.824190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.825760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.827903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.862211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1305/2638] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/projects => generated 359 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:10.866126 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.891755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:10.894930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0cadd-3d2e-4cf3-8510-16233957afe5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.944053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576a9e-6d9f-41af-be06-ff90b5e638ee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.948837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.949440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.949714 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.950006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.950634 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.952565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.956516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.957185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.957464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.957848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] PATH_INFO: `/v3/projects/71157bde6b2a4c5190b21e8a59a12b4f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.959028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71157bde6b2a4c5190b21e8a59a12b4f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.960964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:10.975105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1306/2639] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:10 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:10.978572 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:10.978978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:10.990362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8b096-a2a7-4d40-a058-3643e570e6e8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:10.996172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:10.996769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:10.997044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:10.997310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:10.997954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:10.999599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.010443 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1307/2640] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:11.014487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.015895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71157bde6b2a4c5190b21e8a59a12b4f', 'request_id': 'req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.024555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747be66-7add-4f79-92e2-1111dc69a731 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.024731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71157bde6b2a4c5190b21e8a59a12b4f', 'request_id': 'req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.031239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71157bde6b2a4c5190b21e8a59a12b4f', 'request_id': 'req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.042885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] The token cache is being invalidate because project 71157bde6b2a4c5190b21e8a59a12b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:11.043197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.043930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.045861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1334/2641] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:10 2022] DELETE /identity/v3/projects/71157bde6b2a4c5190b21e8a59a12b4f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:11.049946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.051244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.083291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.084114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.084387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.084688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] PATH_INFO: `/v3/projects/fc2739eaa53e4a71af79580ca83593dc/users/f8ac4cf69ef94b6e9d0ce358df104084/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.100004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2739eaa53e4a71af79580ca83593dc, user_id=f8ac4cf69ef94b6e9d0ce358df104084, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.102749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.117112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11877f-ec67-4cbf-a5f6-55ef39be10a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.118562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1308/2642] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:11 2022] PUT /identity/v3/projects/fc2739eaa53e4a71af79580ca83593dc/users/f8ac4cf69ef94b6e9d0ce358df104084/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:11.122583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.124262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.178690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.179598 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.179878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.180162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] PATH_INFO: `/v3/users/234450fecfc44cc3acc1846acf3cc8fe` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.181432 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=234450fecfc44cc3acc1846acf3cc8fe)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.183367 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.195211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf12d66-2afb-450a-9a70-a7b5f3755ae0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.200166 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.200816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.201105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.201384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.202044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.203722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.217659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1309/2643] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:11.219002 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.229712 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1b4d9-073a-403a-a54c-091be80626af admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.234234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234450fecfc44cc3acc1846acf3cc8fe'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.234603 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.235287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.235578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.235888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] PATH_INFO: `/v3/projects/fc2739eaa53e4a71af79580ca83593dc/users/f8ac4cf69ef94b6e9d0ce358df104084/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.237846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2739eaa53e4a71af79580ca83593dc, user_id=f8ac4cf69ef94b6e9d0ce358df104084, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.239570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:11.240526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea716f-ec5c-4c93-9262-b6986488b6b7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.241583 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234450fecfc44cc3acc1846acf3cc8fe'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.247176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:11.257665 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234450fecfc44cc3acc1846acf3cc8fe'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.260647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1310/2644] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:11 2022] PUT /identity/v3/projects/fc2739eaa53e4a71af79580ca83593dc/users/f8ac4cf69ef94b6e9d0ce358df104084/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:11.264317 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.264668 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.265044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.267951 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234450fecfc44cc3acc1846acf3cc8fe'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.268733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234450fecfc44cc3acc1846acf3cc8fe'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.274876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1335/2645] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:11 2022] DELETE /identity/v3/users/234450fecfc44cc3acc1846acf3cc8fe => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:11.297587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] MFA Rules not processed for user `f8ac4cf69ef94b6e9d0ce358df104084`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:11.298808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.370497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1311/2646] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:11.383946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.385549 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.447878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353248e-4539-4d6f-9e94-1e1bac682b5f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.451777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.452394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.452691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.452956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.462151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.463589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00abe7-2fb3-4463-8291-53c7471307ef service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.473800 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1336/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:11.557820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.559287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.640566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1446c7-12e8-445d-9e16-86d49d69a26e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.647521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.648168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.648449 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.648771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] PATH_INFO: `/v3/projects/cc0c59198381429ba659fc299ad10a55` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.649917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0c59198381429ba659fc299ad10a55)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.651917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.702543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0c59198381429ba659fc299ad10a55', 'request_id': 'req-9b3890e0-5d30-4615-a5da-46dd5de86b7b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.708041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0c59198381429ba659fc299ad10a55', 'request_id': 'req-9b3890e0-5d30-4615-a5da-46dd5de86b7b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.715076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0c59198381429ba659fc299ad10a55', 'request_id': 'req-9b3890e0-5d30-4615-a5da-46dd5de86b7b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.729711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] The token cache is being invalidate because project cc0c59198381429ba659fc299ad10a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:11.729981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.730602 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9b3890e0-5d30-4615-a5da-46dd5de86b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:11.741924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1312/2648] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:11 2022] DELETE /identity/v3/projects/cc0c59198381429ba659fc299ad10a55 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:11.742038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.742038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.742038 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.762736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:11.763941 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.824789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1337/2649] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:11.837522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.837817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.838093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.858750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:11.860044 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.869258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1313/2650] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:11.873861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.884114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abcdf5-061f-4452-a147-0454d72acef3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.888875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.889618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.889919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.890169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.892816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:11.892816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.892816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.905016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1338/2651] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:11.909454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.919077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b0a5b-53e7-430e-a5cb-78e21c73ab09 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.925055 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.925723 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.925954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.926330 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.927066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.929095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.931968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:11.933908 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:11.935265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:11.949959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cdd33-184a-4d1e-a0d5-04e438e689cb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:11.955793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:11.956571 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:11.956953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:11.957320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] PATH_INFO: `/v3/users/fcfe348dd5e44f68b73c4d2a5b37404a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:11.958617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcfe348dd5e44f68b73c4d2a5b37404a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:11.960669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:11.964540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1314/2652] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:11 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:11.972570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.018017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcfe348dd5e44f68b73c4d2a5b37404a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.026240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcfe348dd5e44f68b73c4d2a5b37404a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.027717 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.043385 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaa694-fd99-4f8c-bad6-d07f5d685d83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.044414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcfe348dd5e44f68b73c4d2a5b37404a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.049808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.050411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.050681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.050945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.051599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.053535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.054207 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcfe348dd5e44f68b73c4d2a5b37404a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.054999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2903357e-52c0-44e0-976a-1ebb48928f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcfe348dd5e44f68b73c4d2a5b37404a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.062256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1339/2653] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:11 2022] DELETE /identity/v3/users/fcfe348dd5e44f68b73c4d2a5b37404a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:12.066697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.067113 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.067505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.080427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1315/2654] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:11 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:12.081755 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.084218 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.087199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:12.088361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.098963 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f903b5a-5920-4cfa-85b2-51a06d111ee4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.151040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1340/2655] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:12.153754 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.155255 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.159335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.160079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.160421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.160781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.161501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.163175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.167750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424edde2-097a-4ae8-8214-aa73da2c4fdb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.172885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.173507 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.173783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.173951 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1316/2656] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:12.174701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.174813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] PATH_INFO: `/v3/users/782b5055962b46d69e447a95a08ec972` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.175974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782b5055962b46d69e447a95a08ec972)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.176277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.178177 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.233704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782b5055962b46d69e447a95a08ec972'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.240731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782b5055962b46d69e447a95a08ec972'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.243219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c678-8e00-49d2-9a41-2d35944e39be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.246907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.247518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.247797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.248069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.252907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782b5055962b46d69e447a95a08ec972'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.257223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.258839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d38fc-7b94-44c4-9ebf-2c087f439bc3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.261950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782b5055962b46d69e447a95a08ec972'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.262593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782b5055962b46d69e447a95a08ec972'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.269180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1341/2657] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:12 2022] DELETE /identity/v3/users/782b5055962b46d69e447a95a08ec972 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:12.270008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1317/2658] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:12.270323 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.271775 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.273792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.274202 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.274737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.294114 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:12.295740 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.306453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.332441 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.354552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a18cc6-0255-490b-8c11-d1d5cdbd0a99 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.361330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.361893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1318/2659] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:12.362025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.362210 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.362540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.363190 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.364848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.373585 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.375170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.381678 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1342/2660] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:12.382956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.393926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c792c-34e4-4df2-9fe0-b35561cd1090 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.398937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.399606 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.399873 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.400189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] PATH_INFO: `/v3/projects/b04d7336e6784903bf7b76797ded97cc/users/dad5c61501214a45ac1c17d55bdf7c40/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.415719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04d7336e6784903bf7b76797ded97cc, user_id=dad5c61501214a45ac1c17d55bdf7c40, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.418475 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4edb1-b153-441e-a28b-1546d254d020 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.434594 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4f4f-522d-4b2f-ba82-a3e4b82d9f84 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.444577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1343/2661] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:12 2022] PUT /identity/v3/projects/b04d7336e6784903bf7b76797ded97cc/users/dad5c61501214a45ac1c17d55bdf7c40/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:12.451010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.451298 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.451574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.480530 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] MFA Rules not processed for user `dad5c61501214a45ac1c17d55bdf7c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:12.481872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.485326 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.485882 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.486159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.486439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.557733 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.557961 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1344/2662] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:12.559783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.562506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.570542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1319/2663] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:12.573388 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbf5f-6582-4a4a-b96b-fb27ed4d9680 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.578541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.580170 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.587008 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.589302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.589745 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.590022 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.590869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.592896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.597387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be19b-63e1-4a51-8ed9-eabc331acb0e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.608441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.609389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.609925 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.611262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] PATH_INFO: `/v3/projects/935545f8c1ca424197ce9c86aa23c616` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.612767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935545f8c1ca424197ce9c86aa23c616)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.614855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.633973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1345/2664] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:12 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:12.634719 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.635343 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.673877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935545f8c1ca424197ce9c86aa23c616', 'request_id': 'req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.680983 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935545f8c1ca424197ce9c86aa23c616', 'request_id': 'req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.688415 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935545f8c1ca424197ce9c86aa23c616', 'request_id': 'req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.703990 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] The token cache is being invalidate because project 935545f8c1ca424197ce9c86aa23c616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:12.704295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.705041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.706269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737b8af-f8f2-41f5-b6ef-e47b5f1df125 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.707782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1320/2665] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:12 2022] DELETE /identity/v3/projects/935545f8c1ca424197ce9c86aa23c616 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:12.712191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.714210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.743249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.774230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.774884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.775196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.775438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.776108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.778217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.802220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb8b-b252-4dd6-926c-0ebe8e8bc0a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.805455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1346/2666] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:12 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:12.806479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.807734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.808360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.808596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.809020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.809387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] PATH_INFO: `/v3/users/f8ac4cf69ef94b6e9d0ce358df104084` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.810813 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ac4cf69ef94b6e9d0ce358df104084)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.813262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.821341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177a02-0bdf-4a7a-b22b-8059a2460920 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.825991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.826603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.826896 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.827149 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.827793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.829479 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.843112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1347/2667] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:12.844400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.857915 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba239b1-20d9-4a02-b4e7-a0a0b681d4ae admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:12.864132 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:12.864955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:12.865327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:12.865683 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] PATH_INFO: `/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6/users/e71ec76390284089b13f1359dec733e2/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:12.882080 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58b2fbf72814bd087e22c05ceb68aa6, user_id=e71ec76390284089b13f1359dec733e2, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:12.884756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:12.890390 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ac4cf69ef94b6e9d0ce358df104084'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.898838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1348/2668] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:12 2022] PUT /identity/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6/users/e71ec76390284089b13f1359dec733e2/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:12.899722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ac4cf69ef94b6e9d0ce358df104084'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.900516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ac4cf69ef94b6e9d0ce358df104084'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.905494 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.911941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ac4cf69ef94b6e9d0ce358df104084'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.912862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ac4cf69ef94b6e9d0ce358df104084'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:12.934090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1321/2669] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:12 2022] DELETE /identity/v3/users/f8ac4cf69ef94b6e9d0ce358df104084 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:12.937337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:12.945982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:12.978025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:12.979490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9b03a-1982-4f70-a21a-5bb7b1d7c946 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.025064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ff6ff-eb37-41af-8d32-d68c07f79692 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.030171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.031016 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.031728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.032200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] PATH_INFO: `/v3/projects/d47f2bfdb11549c3b09b7bd20c3722a3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.034125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47f2bfdb11549c3b09b7bd20c3722a3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.036950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.043581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.044268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.045017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.045205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.045922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.047660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.060945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1349/2670] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:13.065443 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.067300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.080850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49d571-1bd9-4bd6-b23c-71ebf2ace780 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.106454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47f2bfdb11549c3b09b7bd20c3722a3', 'request_id': 'req-f2009b34-2390-4aac-9138-18b9ed551489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.112737 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47f2bfdb11549c3b09b7bd20c3722a3', 'request_id': 'req-f2009b34-2390-4aac-9138-18b9ed551489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.112893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:13.119601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47f2bfdb11549c3b09b7bd20c3722a3', 'request_id': 'req-f2009b34-2390-4aac-9138-18b9ed551489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.134535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] The token cache is being invalidate because project d47f2bfdb11549c3b09b7bd20c3722a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:13.134928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.135464 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.137071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1322/2671] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:12 2022] DELETE /identity/v3/projects/d47f2bfdb11549c3b09b7bd20c3722a3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:13.141184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.142844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.152184 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.153486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.153956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.154533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] PATH_INFO: `/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6/users/e71ec76390284089b13f1359dec733e2/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.158054 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58b2fbf72814bd087e22c05ceb68aa6, user_id=e71ec76390284089b13f1359dec733e2, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.162695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907a2f-ca6f-4c09-917b-734590a4bc5c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.175030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1350/2672] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:13 2022] PUT /identity/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6/users/e71ec76390284089b13f1359dec733e2/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:13.185787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.186025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.186355 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.215327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] MFA Rules not processed for user `e71ec76390284089b13f1359dec733e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:13.216669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.218183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009b34-2390-4aac-9138-18b9ed551489 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.290282 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.290936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.291220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.291497 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] PATH_INFO: `/v3/users/31560faf96234981b143f81637610a94` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.292673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31560faf96234981b143f81637610a94)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.294938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.297444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:13.305090 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1351/2673] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:13.306242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.307593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.321091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2504e-5c03-4c39-8b76-ef040d1c7b9e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.325933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.326567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.326848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.327118 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] PATH_INFO: `/v3/projects/fc2739eaa53e4a71af79580ca83593dc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.328377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2739eaa53e4a71af79580ca83593dc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.330381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.345872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31560faf96234981b143f81637610a94'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.352883 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31560faf96234981b143f81637610a94'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.353597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31560faf96234981b143f81637610a94'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.364020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31560faf96234981b143f81637610a94'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.364752 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31560faf96234981b143f81637610a94'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.380669 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2739eaa53e4a71af79580ca83593dc', 'request_id': 'req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.384229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1323/2674] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:13 2022] DELETE /identity/v3/users/31560faf96234981b143f81637610a94 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:13.388053 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.389687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.393258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2739eaa53e4a71af79580ca83593dc', 'request_id': 'req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.401392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2739eaa53e4a71af79580ca83593dc', 'request_id': 'req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.414471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] The token cache is being invalidate because project fc2739eaa53e4a71af79580ca83593dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:13.415028 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.415888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b6152873-dddb-404c-b9e1-5bd6ac9f6c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.418366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1352/2675] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:13 2022] DELETE /identity/v3/projects/fc2739eaa53e4a71af79580ca83593dc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:13.424442 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0eca0-d3c0-4900-9e0c-1deed392e148 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.424967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0eca0-d3c0-4900-9e0c-1deed392e148 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.425446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0eca0-d3c0-4900-9e0c-1deed392e148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.446143 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fec0eca0-d3c0-4900-9e0c-1deed392e148 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:13.447372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fec0eca0-d3c0-4900-9e0c-1deed392e148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.450188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f239f5-ccc1-47c0-a2ee-050e94dcb6b3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.454442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.455063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.455348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.455621 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.457458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.481558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:13.516781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1353/2676] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:13.523410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.523776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.524234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.525568 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.527031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.551991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1324/2677] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:13.553812 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:13.555744 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.566070 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1354/2678] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:13.570255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.582979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936841b-b6dc-4343-a70b-5b93c50bbe97 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.593957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.594674 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.594956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.595252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.595934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:13.596460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.598061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.611190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1325/2679] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:13.615592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.626304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69603cb-f352-403d-b103-e0e6ca943c4a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.631662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.632282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.632617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.632891 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.633545 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.635447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.667660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1355/2680] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:13.672356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.683984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744c70-bd8e-45fe-bfc8-d184e8a0b281 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.688877 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.690647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.702919 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:13.779237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f042e-ab34-4cd7-b9a1-c9008b6f3204 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.784701 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.785358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.785627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.785895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] PATH_INFO: `/v3/projects/01c45a02bb234f039499b38486bb49a5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.787088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c45a02bb234f039499b38486bb49a5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.788974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.793049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.793665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.793941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.794247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.795183 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.797113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.826187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1326/2681] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:13.827247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.838061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0129d-99e4-4018-a738-6356392db067 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.843785 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c45a02bb234f039499b38486bb49a5', 'request_id': 'req-00408954-0d06-4ae2-9aa4-f18ba9620150'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.855271 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c45a02bb234f039499b38486bb49a5', 'request_id': 'req-00408954-0d06-4ae2-9aa4-f18ba9620150'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.872930 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c45a02bb234f039499b38486bb49a5', 'request_id': 'req-00408954-0d06-4ae2-9aa4-f18ba9620150'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.881213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] The token cache is being invalidate because project 01c45a02bb234f039499b38486bb49a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:13.881502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.882185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-00408954-0d06-4ae2-9aa4-f18ba9620150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:13.883699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1356/2682] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:13 2022] DELETE /identity/v3/projects/01c45a02bb234f039499b38486bb49a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:13.888585 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bee3a7-4757-4cab-bbe5-9731ed70bc84 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.888874 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bee3a7-4757-4cab-bbe5-9731ed70bc84 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.889136 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bee3a7-4757-4cab-bbe5-9731ed70bc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.910050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-30bee3a7-4757-4cab-bbe5-9731ed70bc84 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:13.913990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-30bee3a7-4757-4cab-bbe5-9731ed70bc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.926017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.926638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.926916 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.927184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.927896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.932539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.944923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1327/2683] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:13.949098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:13.950898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:13.953426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:13.969628 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918fc8c-5331-4a78-8d19-4b07dae8a7b7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:13.975210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:13.975820 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:13.976097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:13.976366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:13.977050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:13.978663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:13.990921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1328/2684] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:13.992692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1357/2685] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:13.995135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.001385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.001755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.002124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.006800 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea436a14-e4f8-4390-814a-9704f0d26715 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.013071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.014047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.014589 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.015026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] PATH_INFO: `/v3/projects/81ca09c98ecf4c0cab63ba206b66066a/users/14e28329a0074e46814bb607dd4af1d4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.021413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:14.022617 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.035108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1358/2686] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:14.036629 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.038538 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ca09c98ecf4c0cab63ba206b66066a, user_id=14e28329a0074e46814bb607dd4af1d4, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.041357 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705a2ea-8c4b-4818-8a45-62d1dea468c1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.049448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387bc92-bca2-4c9d-9b84-54654d89f910 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.054880 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.055505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.055763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.056040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.056703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:14.057046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.058592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.059697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1329/2687] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:13 2022] PUT /identity/v3/projects/81ca09c98ecf4c0cab63ba206b66066a/users/14e28329a0074e46814bb607dd4af1d4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:14.071142 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1359/2688] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:14.071286 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.071499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.071861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.076079 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.086354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f1cf2-48d1-48bf-b8be-a6af0bb8e6ee admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.104600 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] MFA Rules not processed for user `14e28329a0074e46814bb607dd4af1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:14.105839 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.153235 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.154334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.154864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.155145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.155938 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.158731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.175461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.182750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1330/2689] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:14.187579 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.192428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1360/2690] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:14.198144 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.206953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc046c-d71f-472e-abf7-87a082d6a479 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.265783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b8e96-8ebb-4ae5-9914-f320e4b230d9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.270596 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.271215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.271503 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.271774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.272414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.274380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.274726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.275356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.275745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.276022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.276861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.278859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.301894 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1361/2691] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:14.302995 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.304748 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.309686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1331/2692] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:14.310552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.317939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88a00b-3f1d-46cb-9666-9a1c5adb5e64 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.346829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.372241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc50e7-e424-413d-9d5c-3a8d65b7eaca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.376382 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.377051 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.377165 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.377326 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.377593 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] PATH_INFO: `/v3/users/dad5c61501214a45ac1c17d55bdf7c40` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.377754 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.378042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.378318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.378778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad5c61501214a45ac1c17d55bdf7c40)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.379000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.380618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.380724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.391137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1332/2693] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:14.392121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.402281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d6358-7e67-4f48-9271-8183dbd1d9d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.407879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.408483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.408789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.409068 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.409675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.411604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.432374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad5c61501214a45ac1c17d55bdf7c40'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:14.438896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1333/2694] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:14.438896 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.439731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad5c61501214a45ac1c17d55bdf7c40'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:14.447248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecc88d-c517-4def-96da-6d6d3cb98fd1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.451791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad5c61501214a45ac1c17d55bdf7c40'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:14.451923 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.452483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.452792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.453060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.453745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.455460 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.460158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad5c61501214a45ac1c17d55bdf7c40'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:14.460893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad5c61501214a45ac1c17d55bdf7c40'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:14.466152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1334/2695] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:14.467133 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1362/2696] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:14 2022] DELETE /identity/v3/users/dad5c61501214a45ac1c17d55bdf7c40 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:14.467255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.470417 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.470822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.472224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.483285 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ca44e-2ab4-42ca-a8fd-02b0284f15ca admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.535113 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.536584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38993ac5-681f-4a59-89ed-3cbd1e31c9a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.542055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.542738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.543020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.543324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] PATH_INFO: `/v3/projects/b0ef85da549d4ff690072c860481186a/users/0cc132c5424643af8903ad00fad5da90/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.547429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.548036 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.548314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.548609 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.549256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.550851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d069e-ae95-4eca-84fc-fa169934bc26 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.559913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ef85da549d4ff690072c860481186a, user_id=0cc132c5424643af8903ad00fad5da90, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.560891 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1335/2697] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:14.562207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3890d-9ae7-4d5c-9176-5f7ba795b1a1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.562491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3890d-9ae7-4d5c-9176-5f7ba795b1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.562693 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.562781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3890d-9ae7-4d5c-9176-5f7ba795b1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.578274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1363/2698] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:14 2022] PUT /identity/v3/projects/b0ef85da549d4ff690072c860481186a/users/0cc132c5424643af8903ad00fad5da90/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:14.579134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.582646 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9ee3890d-9ae7-4d5c-9176-5f7ba795b1a1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:14.592238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ee3890d-9ae7-4d5c-9176-5f7ba795b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.612986 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.641771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ececfd-6a5f-409d-8de5-73efc6032477 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.646631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.647290 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.647565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.647831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] PATH_INFO: `/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21/users/0f0f1028aec84448b4f342df42d0fa02/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.657311 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1336/2699] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:14.658578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.658872 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.659148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.662679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0c2d0699b841598c5ef6f1481f0c21, user_id=0f0f1028aec84448b4f342df42d0fa02, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.665387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.680825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1364/2700] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:14 2022] PUT /identity/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21/users/0f0f1028aec84448b4f342df42d0fa02/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:14.681182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.684555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.688198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] MFA Rules not processed for user `0cc132c5424643af8903ad00fad5da90`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:14.689422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.740458 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccebd9-377b-410d-8eb7-9a1b6064ba0a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.744195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.745206 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.745490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.745760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.753588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1337/2701] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:14.757154 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.810697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.812055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.819892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05e975-c9f3-4732-91ca-ee9b6f4e60ae None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.821676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1365/2702] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:14.822445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.824973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.825586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.825862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.826131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.826781 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.828376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.831472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45e9b-1866-4769-b97a-215c93c5bb0e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.836576 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.837179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.837456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.837730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.838360 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.838938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1338/2703] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:14.840359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.843375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.854980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d96c39-a5fc-471f-8c7b-fa66149f758c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:14.862540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:14.863280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.863587 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.863928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] PATH_INFO: `/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21/users/0f0f1028aec84448b4f342df42d0fa02/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.870476 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0c2d0699b841598c5ef6f1481f0c21, user_id=0f0f1028aec84448b4f342df42d0fa02, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:14.873529 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3648a0-7975-4dcc-9958-a356d697dfd6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:14.874390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1366/2704] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:14.878643 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:14.885946 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.896588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1339/2705] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:14 2022] PUT /identity/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21/users/0f0f1028aec84448b4f342df42d0fa02/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:14.905631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:14.905914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:14.906185 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:14.932456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:14.941325 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] MFA Rules not processed for user `0f0f1028aec84448b4f342df42d0fa02`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:14.942577 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:14.955918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244c7c5-88e9-4f80-9212-64d2aa06f960 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.022558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.023301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.023833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.024101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.024849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.027115 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.034807 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1340/2706] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:15.043601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.047181 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.055502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1367/2707] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:14 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:15.059876 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.071005 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccd103-4aa5-4bf3-8729-536690445f0a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.075760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.076396 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.077020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.077020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.077770 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.079548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.092804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1368/2708] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:15.094632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.106984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54d468-f26e-49bc-a181-7c8515c5b9a2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.114681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.115340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.115604 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.115871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] PATH_INFO: `/v3/projects/9c38e7d4722a4e27a931999c9359dedc/users/61be27fc4fe84191b2d0c1eafe3f1471/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.122630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbad7a-6bc1-420a-9c82-fc88bfba1283 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.126429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.127049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.127333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.127607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.133126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c38e7d4722a4e27a931999c9359dedc, user_id=61be27fc4fe84191b2d0c1eafe3f1471, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.135763 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.137222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.138789 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.150175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1341/2709] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:15.151790 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.164580 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1369/2710] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:15 2022] PUT /identity/v3/projects/9c38e7d4722a4e27a931999c9359dedc/users/61be27fc4fe84191b2d0c1eafe3f1471/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:15.164879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.165467 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.187007 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf185e1-8df4-4c2a-af69-849335439f2e service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.247525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dec195-5f52-4d51-928c-9cdfdb53818e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.253448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.254129 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.254465 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.254826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.256301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.257322 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.266532 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.267258 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.267540 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.267818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] PATH_INFO: `/v3/projects/b04d7336e6784903bf7b76797ded97cc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.268588 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1370/2711] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:15.269167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04d7336e6784903bf7b76797ded97cc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.271337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.273114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.289266 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bccd-f1b2-4341-9d1b-f25320d79315 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.294242 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.295256 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.295539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.295840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] PATH_INFO: `/v3/projects/9c38e7d4722a4e27a931999c9359dedc/users/61be27fc4fe84191b2d0c1eafe3f1471/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.297907 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c38e7d4722a4e27a931999c9359dedc, user_id=61be27fc4fe84191b2d0c1eafe3f1471, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.300901 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0f69b-ca47-419b-8e4c-0684b008d580 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.317510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1371/2712] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:15 2022] PUT /identity/v3/projects/9c38e7d4722a4e27a931999c9359dedc/users/61be27fc4fe84191b2d0c1eafe3f1471/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:15.328684 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.328975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.329251 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.330985 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:15.347376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04d7336e6784903bf7b76797ded97cc', 'request_id': 'req-632a5dbe-67f3-4800-8558-08ad9f8be6e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.354410 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04d7336e6784903bf7b76797ded97cc', 'request_id': 'req-632a5dbe-67f3-4800-8558-08ad9f8be6e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.361739 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04d7336e6784903bf7b76797ded97cc', 'request_id': 'req-632a5dbe-67f3-4800-8558-08ad9f8be6e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.363711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] MFA Rules not processed for user `61be27fc4fe84191b2d0c1eafe3f1471`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:15.365917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.379521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] The token cache is being invalidate because project b04d7336e6784903bf7b76797ded97cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:15.380354 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.380736 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.382611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1342/2713] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:15 2022] DELETE /identity/v3/projects/b04d7336e6784903bf7b76797ded97cc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:15.386484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.387978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.394920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:15.455688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a5dbe-67f3-4800-8558-08ad9f8be6e2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.460889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.461596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.461928 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.462307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] PATH_INFO: `/v3/users/e71ec76390284089b13f1359dec733e2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.463522 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71ec76390284089b13f1359dec733e2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.465567 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.467196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1372/2714] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:15.479233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.481561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.529380 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71ec76390284089b13f1359dec733e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.539454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71ec76390284089b13f1359dec733e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.540611 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71ec76390284089b13f1359dec733e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.551058 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71ec76390284089b13f1359dec733e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.552586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71ec76390284089b13f1359dec733e2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:15.557236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33834ff2-63f2-4d72-96d9-640426117dd9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:15.561100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:15.561768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:15.562131 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:15.562405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:15.564586 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.580634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:15.580634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:15.593575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1343/2715] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:15 2022] DELETE /identity/v3/users/e71ec76390284089b13f1359dec733e2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:15.608592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1373/2716] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:15.938726 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:15.940299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:15.998167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:16.032195 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b7b97-4cae-4482-ab9a-54b0f6c7ceee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.037315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.037649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.037935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.038169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] PATH_INFO: `/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.042572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58b2fbf72814bd087e22c05ceb68aa6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.042572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.043985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.045819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.059860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a220c-bad5-4aab-b56a-dbf88632cf20 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.064964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.065635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.065917 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.066191 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] PATH_INFO: `/v3/users/0cc132c5424643af8903ad00fad5da90` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.067333 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc132c5424643af8903ad00fad5da90)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.069324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.099954 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58b2fbf72814bd087e22c05ceb68aa6', 'request_id': 'req-e6642ac6-f0c1-4746-8216-3bb18574e706'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.106679 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58b2fbf72814bd087e22c05ceb68aa6', 'request_id': 'req-e6642ac6-f0c1-4746-8216-3bb18574e706'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.115472 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58b2fbf72814bd087e22c05ceb68aa6', 'request_id': 'req-e6642ac6-f0c1-4746-8216-3bb18574e706'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.130495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] The token cache is being invalidate because project d58b2fbf72814bd087e22c05ceb68aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:16.130844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.131599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e6642ac6-f0c1-4746-8216-3bb18574e706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.132498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc132c5424643af8903ad00fad5da90'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.135252 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1344/2717] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:15 2022] DELETE /identity/v3/projects/d58b2fbf72814bd087e22c05ceb68aa6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:16.139899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580ce0-347d-497c-bf6b-45e8e67b801a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.140074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc132c5424643af8903ad00fad5da90'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.140196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580ce0-347d-497c-bf6b-45e8e67b801a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.140469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580ce0-347d-497c-bf6b-45e8e67b801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.158840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc132c5424643af8903ad00fad5da90'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.163837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4e580ce0-347d-497c-bf6b-45e8e67b801a None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:16.165136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e580ce0-347d-497c-bf6b-45e8e67b801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.167357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc132c5424643af8903ad00fad5da90'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.168217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2b0c268a-dc92-42b8-ae6e-ca3e41df2186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc132c5424643af8903ad00fad5da90'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.175563 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1374/2718] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:16 2022] DELETE /identity/v3/users/0cc132c5424643af8903ad00fad5da90 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:16.180140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.180425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.180708 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.203181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:16.204570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.231003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:16.238880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1345/2719] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:16.245324 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.245720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.246077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.266767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:16.267987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.281282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1375/2720] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:16.291504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.293172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.343742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1346/2721] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:16.344945 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.355321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666570f-1aa1-41ae-b3d3-ece20b8692cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.360186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.360803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759af094-ca56-4dbe-bcca-2922c81e28d3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.360939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.361129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.361430 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.362031 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:16.362338 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.364135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.364648 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.365259 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.365533 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.365799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.366836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:16.378829 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.379039 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1347/2722] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:16.380274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.383371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.391119 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1376/2723] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:16.396189 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c89bc-3c72-489e-8a0b-17e47a88ef3c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.401685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.402321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.402596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.402862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.403526 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.405444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.443453 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1348/2724] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:16.447869 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.460145 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c00a-c0ef-43aa-818a-02a625f5f7ea service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.528759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.529381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.529660 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.529932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.530574 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.533988 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.565814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1377/2725] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:16.567780 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.579884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee84c9c-5e58-4084-b235-4b9842382753 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.584640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.585598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.585880 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.586147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.586809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.588436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.602064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1349/2726] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:16.606258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.625209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc094aa9-1bc6-4703-8d77-afd0a98a8448 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.631002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.631690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.632040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.632716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.633595 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.635221 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e22ff-49c9-4512-83cb-5f326fac24e9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.646726 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1378/2727] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:16.650667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.661787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f43a-3e55-4c57-8274-d0778c4505bf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.667179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.667917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.668256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.668622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] PATH_INFO: `/v3/projects/34f547f2bae94d50a7ad56ba49d0f8b9/users/d4876dc8c92f4788bd3143b48f660f7a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.684713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f547f2bae94d50a7ad56ba49d0f8b9, user_id=d4876dc8c92f4788bd3143b48f660f7a, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.687482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.701988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1350/2728] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:16 2022] PUT /identity/v3/projects/34f547f2bae94d50a7ad56ba49d0f8b9/users/d4876dc8c92f4788bd3143b48f660f7a/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:16.716454 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.716838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.717107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.719120 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.720921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.757661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] MFA Rules not processed for user `d4876dc8c92f4788bd3143b48f660f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:16.758928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.805046 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6842-ea5a-4388-a1af-8dbf04c24d17 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.807687 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:16.809947 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.810654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.810997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.811333 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] PATH_INFO: `/v3/projects/b0ef85da549d4ff690072c860481186a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.812531 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ef85da549d4ff690072c860481186a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.814518 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.834486 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1379/2729] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:16.835328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.836796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.850096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf587715-1abc-4e12-b4f7-48b2a6671e8f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:16.854634 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:16.855262 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:16.855541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:16.855811 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] PATH_INFO: `/v3/users/14e28329a0074e46814bb607dd4af1d4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:16.856970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e28329a0074e46814bb607dd4af1d4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:16.858905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:16.865552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ef85da549d4ff690072c860481186a', 'request_id': 'req-02adfd8e-ce59-42f5-a0af-a9014c592b57'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.871030 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ef85da549d4ff690072c860481186a', 'request_id': 'req-02adfd8e-ce59-42f5-a0af-a9014c592b57'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.877427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ef85da549d4ff690072c860481186a', 'request_id': 'req-02adfd8e-ce59-42f5-a0af-a9014c592b57'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.893163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] The token cache is being invalidate because project b0ef85da549d4ff690072c860481186a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:16.893450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.894105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.895911 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1351/2730] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:16 2022] DELETE /identity/v3/projects/b0ef85da549d4ff690072c860481186a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:16.896797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.898123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.912759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e28329a0074e46814bb607dd4af1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.920140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e28329a0074e46814bb607dd4af1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.929636 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:16.932752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e28329a0074e46814bb607dd4af1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.941830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e28329a0074e46814bb607dd4af1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.942875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e28329a0074e46814bb607dd4af1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:16.949567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1380/2731] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:16 2022] DELETE /identity/v3/users/14e28329a0074e46814bb607dd4af1d4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:16.950459 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:16.951768 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:16.965184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adfd8e-ce59-42f5-a0af-a9014c592b57 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.015162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f30d-90b8-4d9d-bea9-cf671bc67d53 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.019892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.020373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.020535 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.020820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.021017 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.021135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] PATH_INFO: `/v3/users/61be27fc4fe84191b2d0c1eafe3f1471` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.021297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.021574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.022211 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.022342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61be27fc4fe84191b2d0c1eafe3f1471)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.024116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d048209-9897-41d6-b4fc-3c7d5b8ec7de admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.024237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.054827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1352/2732] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:16 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:17.054827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.054827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.054827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.073088 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:17.074306 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.076932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61be27fc4fe84191b2d0c1eafe3f1471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.082351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.084783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61be27fc4fe84191b2d0c1eafe3f1471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.099381 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61be27fc4fe84191b2d0c1eafe3f1471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.109099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61be27fc4fe84191b2d0c1eafe3f1471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.109892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61be27fc4fe84191b2d0c1eafe3f1471'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.116060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1381/2733] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:16 2022] DELETE /identity/v3/users/61be27fc4fe84191b2d0c1eafe3f1471 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:17.116824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.118146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.138404 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1353/2734] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:17.151121 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.153098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.191810 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e710ad-0cce-4af0-ac16-a43e91266132 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.198154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.199021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.199408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.199756 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.200435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.202890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.216095 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7789-3b9e-4319-a7fc-2f9202814321 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.227573 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1382/2735] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:17.230917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.233729 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.245241 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc195c-8e27-4898-837e-2da86c9ec697 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.250707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.251703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.251703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.251962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.252485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.254124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.265935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1383/2736] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:17.269614 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.270741 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.271525 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.271895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.272272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.284804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.286368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724be84-fc68-4a90-a360-2e8c821819ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.286500 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.293002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.294001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.294341 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.295021 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.296196 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.297692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1354/2737] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:17.297820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc453e2-599a-4716-a644-1afad3c087d9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.308833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1384/2738] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:17.313208 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.323590 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde72-d0b5-4475-a906-e850588f5c87 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.329474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.330819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.331108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.331392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] PATH_INFO: `/v3/projects/0ef5698e114649218291742d5f3992a4/users/7b73d620c3b3499d8119c35b4c965a7c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.349799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef5698e114649218291742d5f3992a4, user_id=7b73d620c3b3499d8119c35b4c965a7c, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.352678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.369018 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1355/2739] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:17 2022] PUT /identity/v3/projects/0ef5698e114649218291742d5f3992a4/users/7b73d620c3b3499d8119c35b4c965a7c/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:17.380698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.381052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.381272 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.427752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] MFA Rules not processed for user `7b73d620c3b3499d8119c35b4c965a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:17.429527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.434537 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.437461 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.511384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1385/2740] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:17.516335 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.529751 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b05ef-b8b0-47e2-9508-eb98f242679f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.535158 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.536171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.536171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.536423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] PATH_INFO: `/v3/projects/9c38e7d4722a4e27a931999c9359dedc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.537575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c38e7d4722a4e27a931999c9359dedc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.539466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.567324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.606179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c38e7d4722a4e27a931999c9359dedc', 'request_id': 'req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.613745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c38e7d4722a4e27a931999c9359dedc', 'request_id': 'req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.614010 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73cf1-75e8-4cbb-8479-e12c15a8cb52 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.620413 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c38e7d4722a4e27a931999c9359dedc', 'request_id': 'req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.636799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] The token cache is being invalidate because project 9c38e7d4722a4e27a931999c9359dedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:17.637680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.638361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.640136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1356/2741] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:17 2022] DELETE /identity/v3/projects/9c38e7d4722a4e27a931999c9359dedc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:17.641612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.643064 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.673952 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.674165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.674750 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.675632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.675841 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.676555 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.678541 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cc0d3-ccd1-4835-96c3-617a59277cc6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.709791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1386/2742] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:17.709791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908779-dba2-4483-8536-70be99414c88 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.709791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908779-dba2-4483-8536-70be99414c88 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.709791 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908779-dba2-4483-8536-70be99414c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.711383 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4ed2f-2c37-47eb-a592-58d5c13a813f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.727828 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-17908779-dba2-4483-8536-70be99414c88 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:17.729071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17908779-dba2-4483-8536-70be99414c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.769687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.770498 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.770778 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.771054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] PATH_INFO: `/v3/projects/81ca09c98ecf4c0cab63ba206b66066a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.772343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ca09c98ecf4c0cab63ba206b66066a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.774303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.791308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1387/2743] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:17.792231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17908779-dba2-4483-8536-70be99414c88 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.793705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17908779-dba2-4483-8536-70be99414c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.806394 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17908779-dba2-4483-8536-70be99414c88 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.811984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.812765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.813043 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.813308 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.813937 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.815904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adb4d1-9fc6-4f23-aa7b-71dc07280a4f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.823547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ca09c98ecf4c0cab63ba206b66066a', 'request_id': 'req-3c16d0e9-931a-473a-a501-518d080fdf25'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.829874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ca09c98ecf4c0cab63ba206b66066a', 'request_id': 'req-3c16d0e9-931a-473a-a501-518d080fdf25'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.836398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ca09c98ecf4c0cab63ba206b66066a', 'request_id': 'req-3c16d0e9-931a-473a-a501-518d080fdf25'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.841456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1388/2744] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/users => generated 299 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:17.841456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.841456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.841456 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.853278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] The token cache is being invalidate because project 81ca09c98ecf4c0cab63ba206b66066a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:17.853506 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.854141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:17.855685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1357/2745] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:17 2022] DELETE /identity/v3/projects/81ca09c98ecf4c0cab63ba206b66066a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:17.856429 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.857798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.861802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:17.862991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.905306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.924893 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16d0e9-931a-473a-a501-518d080fdf25 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.925415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1389/2746] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:17.926244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.927551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:17.933099 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.933720 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.933999 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.934272 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.934958 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.936619 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.939862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b9f27-4f98-426b-aad8-23f8b50e19e4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.942063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:17.945616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.946245 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.946527 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.946802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] PATH_INFO: `/v3/users/0f0f1028aec84448b4f342df42d0fa02` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.947696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1358/2747] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:17.948683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.951214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0f1028aec84448b4f342df42d0fa02)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.953280 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.959786 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a320-5bcf-48b5-bc34-f0da9c4d3ad9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.964766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.965360 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.965640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.965913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:17.966556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:17.966853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:17.968433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:17.982287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1359/2748] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:17.983082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:17.993667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186ffcb-b177-4d92-bb39-0a6091eb74cb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:17.998375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:17.999180 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:17.999362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:17.999630 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] PATH_INFO: `/v3/projects/d5e4fe22264b4d49b608d28e940ff375/users/70521e7c4b024fc6b563641c3418e5f8/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.006217 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0f1028aec84448b4f342df42d0fa02'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.013342 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0f1028aec84448b4f342df42d0fa02'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.014303 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e4fe22264b4d49b608d28e940ff375, user_id=70521e7c4b024fc6b563641c3418e5f8, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.017006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.026176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0f1028aec84448b4f342df42d0fa02'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.032716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1360/2749] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:17 2022] PUT /identity/v3/projects/d5e4fe22264b4d49b608d28e940ff375/users/70521e7c4b024fc6b563641c3418e5f8/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:18.033487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.034165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0f1028aec84448b4f342df42d0fa02'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.034783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0f1028aec84448b4f342df42d0fa02'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.042151 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1390/2750] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:17 2022] DELETE /identity/v3/users/0f0f1028aec84448b4f342df42d0fa02 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:18.043024 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.044433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.108873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e54e6-95d6-464e-ba3d-417d0b2b4518 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.114662 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21047d6-24ae-42ed-b821-cd5d4dcf9b7d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.115750 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.116496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.116844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.117090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.117988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.119861 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.120022 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.120540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.120942 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.121227 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.122127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.123872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.136562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1391/2751] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:18.141002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.152516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0c9b5-28e9-42d5-9465-0755c195b70f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.158949 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1361/2752] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:18.159135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.159135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.159135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.159135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] PATH_INFO: `/v3/projects/d5e4fe22264b4d49b608d28e940ff375/users/70521e7c4b024fc6b563641c3418e5f8/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.160440 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.160959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e4fe22264b4d49b608d28e940ff375, user_id=70521e7c4b024fc6b563641c3418e5f8, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.162689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.163603 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8d190-eaf2-440e-93f5-2674bf4129de admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.175840 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1392/2753] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:18 2022] PUT /identity/v3/projects/d5e4fe22264b4d49b608d28e940ff375/users/70521e7c4b024fc6b563641c3418e5f8/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:18.186003 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.186234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.186468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.228168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] MFA Rules not processed for user `70521e7c4b024fc6b563641c3418e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:18.229627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.245549 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540600-0570-45b5-bc2b-dc0d56a617eb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.265117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:18.299649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:18.310659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.312115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.312708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.313096 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.314421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.317499 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.319513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1393/2754] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:18.333078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.334991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.346779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1362/2755] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:18.347698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.357082 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.370247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39011-2291-485d-a0f8-6981c85f429c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.375174 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.375850 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.376128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.376427 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] PATH_INFO: `/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.377581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0c2d0699b841598c5ef6f1481f0c21)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.379524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.402199 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43826845-9e61-4199-9dd6-964b1f3e3d9b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.406296 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.406926 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.407229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.407505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.409436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.427802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0c2d0699b841598c5ef6f1481f0c21', 'request_id': 'req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.433591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0c2d0699b841598c5ef6f1481f0c21', 'request_id': 'req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.439889 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0c2d0699b841598c5ef6f1481f0c21', 'request_id': 'req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.452851 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] The token cache is being invalidate because project 7c0c2d0699b841598c5ef6f1481f0c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:18.453363 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.453920 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:18.455495 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1363/2756] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:18 2022] DELETE /identity/v3/projects/7c0c2d0699b841598c5ef6f1481f0c21 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:18.459652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.461027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.470534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.471981 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225572e-96af-4484-ac7d-3a58f98e3336 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.481052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1394/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:18.484025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfadf-4162-4a11-8c11-05aa6873ebf1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.484300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfadf-4162-4a11-8c11-05aa6873ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.484715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfadf-4162-4a11-8c11-05aa6873ebf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.507521 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a05bfadf-4162-4a11-8c11-05aa6873ebf1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:18.508790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a05bfadf-4162-4a11-8c11-05aa6873ebf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.532279 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc5fdb-7ace-4f7d-a271-d6b4ae5816a1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.539566 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.540259 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.540631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.540969 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.541698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.543398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.554992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1364/2758] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:18.559128 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.570936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5e0ce-f0d7-4f3d-899b-e984db02bd0f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.575649 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.576368 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.576688 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.576966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.578071 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.579862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1395/2759] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:18.579987 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.587743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.588030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.588303 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.599487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1365/2760] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:18.603060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.609292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:18.611182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.614239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635739b-b1bd-43db-bb61-dc08ef42cde9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.619248 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.619853 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.620142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.620418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] PATH_INFO: `/v3/projects/4fef28dcd9cb43f48f082f2342419268/users/1161d6181bce4c31bbb4ec8bedd2f4bc/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.623766 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1396/2761] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:18.624674 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.634916 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc828e-bfa0-440f-a796-7eeb7645b973 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.640101 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.640752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.641065 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.641339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.641997 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:18.642301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.643913 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.645746 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fef28dcd9cb43f48f082f2342419268, user_id=1161d6181bce4c31bbb4ec8bedd2f4bc, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.649738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856a69f-22c3-46b3-954b-f659c91793bf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.653324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:18.656977 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1397/2762] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:18.661376 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.670028 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1366/2763] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:18 2022] PUT /identity/v3/projects/4fef28dcd9cb43f48f082f2342419268/users/1161d6181bce4c31bbb4ec8bedd2f4bc/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:18.672679 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e3473-1bff-4384-8e6e-15f5a3c2943f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.682027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.682318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.682575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.712960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] MFA Rules not processed for user `1161d6181bce4c31bbb4ec8bedd2f4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:18.714190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:18.734696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:18.739628 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.740224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.740524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.740799 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.741439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.743357 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.784547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1398/2764] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:18.787551 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.823321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1367/2765] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:18.827607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.846517 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7b8a7-42a4-4028-90b6-1b176a8b6fb0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.863843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818479d-e155-4971-af7b-84f37e70ad8a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.871351 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.872114 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.872732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.873069 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.873897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.877732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.907059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1399/2766] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:18.910888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.914804 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.915432 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.915713 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.916277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.916948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.918955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.921361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fe681-f06d-46d1-b646-b9045bc14192 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:18.926721 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:18.927359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:18.927641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:18.927968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:18.928652 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:18.930257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:18.940808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1400/2767] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:18.944790 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.947398 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1368/2768] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:18.951009 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:18.954821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310f6ee-f66e-43a4-b533-2675c730e194 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.011925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.012564 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.012848 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.013117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.013734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0a405-1d90-4331-b6d2-ff2dfa2a73b4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.013862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.015340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.018660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.019265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.019572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.019837 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.020539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.020673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.022511 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.028728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1401/2769] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:19.030082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.040612 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f529ee-d74a-47c3-853c-f088004c10c8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.045699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.046340 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.046616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.046884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] PATH_INFO: `/v3/projects/3879de435d8544798a945a169a5083a8/users/8ee993ca931649e899fd3f0054279b49/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.047462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1369/2770] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:18 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:19.051347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.061433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f56ff-923b-47b6-955f-bbf1516c653b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.062490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3879de435d8544798a945a169a5083a8, user_id=8ee993ca931649e899fd3f0054279b49, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.065276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.066475 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.067052 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.067387 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.067665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.068316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.070309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a9f7f-5768-44a0-a295-7585e68a7066 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.080274 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1402/2771] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:19 2022] PUT /identity/v3/projects/3879de435d8544798a945a169a5083a8/users/8ee993ca931649e899fd3f0054279b49/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:19.085345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1370/2772] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:19.088621 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.091478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.091788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.092061 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.119687 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] MFA Rules not processed for user `8ee993ca931649e899fd3f0054279b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:19.120866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:19.129912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.155835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc860b-3f0d-40b9-a985-4778b39dba63 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.161464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.162553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.162830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.163099 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] PATH_INFO: `/v3/projects/908064553c224d46a7de5e08d65754c7/users/4e604a2190cf4d2c8c040ababfb30d23/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.178577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908064553c224d46a7de5e08d65754c7, user_id=4e604a2190cf4d2c8c040ababfb30d23, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.181306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.188615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1371/2773] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:19.193213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.201783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1403/2774] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:19 2022] PUT /identity/v3/projects/908064553c224d46a7de5e08d65754c7/users/4e604a2190cf4d2c8c040ababfb30d23/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:19.204025 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1279f57-8365-40b3-8265-a0a73b588231 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.205975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.263981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.264622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.264909 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.265331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.266160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.268098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.268661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900001a7-a89e-4ed7-ab88-c07dd8a3cae5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.273363 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.273970 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.274304 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.274548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.275211 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.276808 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.286664 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1404/2775] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:19.291457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.297492 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1372/2776] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:19 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:19.301512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.302182 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f5591-5c1d-4ebe-b2c5-bf5b9e82aee5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.366125 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5e90-60ed-4c40-8a09-5caed12e9126 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.371973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.372552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.372846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.373115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.373809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.375856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.400816 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1373/2777] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:19 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:19.405091 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.415935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979082d9-4491-4cf6-8ac7-7e978748f401 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.421350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.421991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.422302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.422584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.423238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.424910 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.437010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1374/2778] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:19.441118 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.451675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f696ba-85d4-494c-84cc-8d7ec8600ce4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.456818 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.457565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.457879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.458159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] PATH_INFO: `/v3/projects/6bdec729c57545ed975ced76d00b0ec1/users/dad4d3b8789f4850b486197c9a6248a1/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.470570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.473785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdec729c57545ed975ced76d00b0ec1, user_id=dad4d3b8789f4850b486197c9a6248a1, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.476846 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.480609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.487991 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.488982 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.489400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.489776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] PATH_INFO: `/v3/projects/908064553c224d46a7de5e08d65754c7/users/4e604a2190cf4d2c8c040ababfb30d23/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.492410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=908064553c224d46a7de5e08d65754c7, user_id=4e604a2190cf4d2c8c040ababfb30d23, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.493535 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1375/2779] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:19 2022] PUT /identity/v3/projects/6bdec729c57545ed975ced76d00b0ec1/users/dad4d3b8789f4850b486197c9a6248a1/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:19.495448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71829b-70e4-4942-8a78-f552dfb7ae4d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.497454 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.507346 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1405/2780] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:19 2022] PUT /identity/v3/projects/908064553c224d46a7de5e08d65754c7/users/4e604a2190cf4d2c8c040ababfb30d23/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:19.518509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.518825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.519108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.547105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] MFA Rules not processed for user `4e604a2190cf4d2c8c040ababfb30d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:19.548400 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:19.561080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e43d56-9047-4973-b441-1eaf3e4f751c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.618924 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1406/2781] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:19.623275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.623965 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.624310 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.625098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.625334 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.626940 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.630602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.632017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:19.640847 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1376/2782] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:19.641992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.652359 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064434c-ba39-4293-9f14-cf4613522084 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.657792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.658447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.658731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.659005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] PATH_INFO: `/v3/projects/6bdec729c57545ed975ced76d00b0ec1/users/dad4d3b8789f4850b486197c9a6248a1/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.661092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdec729c57545ed975ced76d00b0ec1, user_id=dad4d3b8789f4850b486197c9a6248a1, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.663782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178584c-2c51-4a0a-9f46-b952e4d13629 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.674570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1377/2783] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:19 2022] PUT /identity/v3/projects/6bdec729c57545ed975ced76d00b0ec1/users/dad4d3b8789f4850b486197c9a6248a1/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:19.685171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.685481 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.685734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.688851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fa154-a929-45fb-b9b9-002466a9f1dd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.714187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] MFA Rules not processed for user `dad4d3b8789f4850b486197c9a6248a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:19.715421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:19.739282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.739912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.740174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.740440 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.784576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1378/2784] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:19.790107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.796340 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.797912 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:19.806102 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.807561 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.809656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c509c-0abb-455f-9a1c-9efce1f63b41 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.813693 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.814309 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.814588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.814859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.816697 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1407/2785] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:19.821056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:19.826624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.828393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.838348 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1379/2786] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:19.884267 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:19.893783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56be8f-ddd2-4268-bac3-5f5a1da35aca service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:19.897410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:19.898017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:19.898291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:19.898559 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:19.957600 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:19.959039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:19.967909 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1408/2787] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:20.352672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:20.362601 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70e22-285c-4e49-a57a-b37f505dc360 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:20.367148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:20.367798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:20.368026 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:20.368696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:20.370175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:20.432270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:20.433749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:20.442787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1380/2788] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:21.002050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:21.010895 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6d076-ef27-4d10-a1c6-8a8f0c56862f service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:21.014715 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:21.015387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:21.015783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:21.016095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:21.017974 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:21.099075 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:21.100536 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44767dc-ff8e-4ee3-9ac6-26a9ab850918 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:21.110243 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1409/2789] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:23.396422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:23.462267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de7a0f-66cc-419c-9b84-83a01c95f9ef service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:23.467328 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:23.467957 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:23.468238 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:23.468530 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] PATH_INFO: `/v3/users/8ee993ca931649e899fd3f0054279b49` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:23.469801 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ee993ca931649e899fd3f0054279b49)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:23.471760 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:23.521392 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ee993ca931649e899fd3f0054279b49'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:23.528823 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ee993ca931649e899fd3f0054279b49'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:23.529512 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ee993ca931649e899fd3f0054279b49'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:23.538396 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ee993ca931649e899fd3f0054279b49'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:23.539011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ee993ca931649e899fd3f0054279b49'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:23.557005 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1381/2790] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:23 2022] DELETE /identity/v3/users/8ee993ca931649e899fd3f0054279b49 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:23.561681 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:23.562174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:23.562282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:23.584822 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:23.586169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:23.652587 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1410/2791] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:23.662617 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:23.664057 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:23.679878 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:23.727220 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e952d-5db8-4487-8649-1423a77e7d5c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:23.730968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:23.731581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:23.731862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:23.732129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:23.742616 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:23.744051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:23.754852 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1382/2792] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:24.255110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:24.257300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:24.278752 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:24.293394 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:24.296236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:24.309895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec11267-3de5-464b-a9b8-e492190ecb49 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:24.315469 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:24.316115 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:24.316393 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:24.316769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] PATH_INFO: `/v3/projects/3879de435d8544798a945a169a5083a8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:24.317899 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3879de435d8544798a945a169a5083a8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:24.319934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:24.337496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79ec16-164e-4054-aafc-1b2386648a6b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:24.341122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:24.341738 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:24.342011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:24.342276 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:24.343999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:24.370491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3879de435d8544798a945a169a5083a8', 'request_id': 'req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.375991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3879de435d8544798a945a169a5083a8', 'request_id': 'req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.382770 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3879de435d8544798a945a169a5083a8', 'request_id': 'req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.397246 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] The token cache is being invalidate because project 3879de435d8544798a945a169a5083a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:24.397877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.398198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.399861 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1383/2793] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:24 2022] DELETE /identity/v3/projects/3879de435d8544798a945a169a5083a8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:24.403675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:24.405162 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:24.410229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:24.411688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:24.421418 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1411/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:24.470979 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f1cd-3c33-4887-88a1-b0a4a74a07ba admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:24.475742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:24.476371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:24.476694 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:24.476962 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] PATH_INFO: `/v3/users/dad4d3b8789f4850b486197c9a6248a1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:24.478112 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad4d3b8789f4850b486197c9a6248a1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:24.480015 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:24.522103 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:24.529558 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad4d3b8789f4850b486197c9a6248a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.538874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad4d3b8789f4850b486197c9a6248a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.539556 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad4d3b8789f4850b486197c9a6248a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.547373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad4d3b8789f4850b486197c9a6248a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.548104 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6b78e619-b971-475a-8ed5-1e4ed132872f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad4d3b8789f4850b486197c9a6248a1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.566835 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1384/2795] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:24 2022] DELETE /identity/v3/users/dad4d3b8789f4850b486197c9a6248a1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:24.842124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:24.853138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09684-1758-42e0-ae15-9d2a7f9015b7 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:24.912219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:24.912804 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:24.913071 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:24.913332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] PATH_INFO: `/v3/projects/6bdec729c57545ed975ced76d00b0ec1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:24.914422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bdec729c57545ed975ced76d00b0ec1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:24.916302 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:24.965078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bdec729c57545ed975ced76d00b0ec1', 'request_id': 'req-978986cc-a305-41f3-a9e4-828cbd9acab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.974047 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bdec729c57545ed975ced76d00b0ec1', 'request_id': 'req-978986cc-a305-41f3-a9e4-828cbd9acab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.980952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bdec729c57545ed975ced76d00b0ec1', 'request_id': 'req-978986cc-a305-41f3-a9e4-828cbd9acab5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.987856 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:24.993244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] The token cache is being invalidate because project 6bdec729c57545ed975ced76d00b0ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:24.993636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.994463 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:24.999831 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1412/2796] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:24 2022] DELETE /identity/v3/projects/6bdec729c57545ed975ced76d00b0ec1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:25.004245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db19f-c814-4bc0-bb53-864ebeac9471 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.004245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db19f-c814-4bc0-bb53-864ebeac9471 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.004245 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db19f-c814-4bc0-bb53-864ebeac9471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.005548 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.007875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.023478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-431db19f-c814-4bc0-bb53-864ebeac9471 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:25.024788 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-431db19f-c814-4bc0-bb53-864ebeac9471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.071967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978986cc-a305-41f3-a9e4-828cbd9acab5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.077247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.077872 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.078138 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.078408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] PATH_INFO: `/v3/users/d4876dc8c92f4788bd3143b48f660f7a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.079515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4876dc8c92f4788bd3143b48f660f7a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.081513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.085782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1385/2797] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:25.093903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.094213 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.094488 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.113886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:25.115219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.123153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1386/2798] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:25.127301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.128667 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4876dc8c92f4788bd3143b48f660f7a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.135546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4876dc8c92f4788bd3143b48f660f7a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.136152 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6661-3b90-42d9-8953-bc02e7081cb6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.140845 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.141433 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.141716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.141984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.142691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:25.143019 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.144655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.147777 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4876dc8c92f4788bd3143b48f660f7a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.157219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1387/2799] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:25.158639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4876dc8c92f4788bd3143b48f660f7a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.159283 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-40eac418-5386-4a2a-b41e-3d18862e8c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4876dc8c92f4788bd3143b48f660f7a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.161402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.163273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.165450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1413/2800] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:25 2022] DELETE /identity/v3/users/d4876dc8c92f4788bd3143b48f660f7a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:25.169947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.170230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.170498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.175973 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675e43f-33a0-448a-8c6f-3b2cc6842428 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.189238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:25.190476 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.233314 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.233915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.234190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.234459 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.235089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.237090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.249446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1414/2801] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:25.259168 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.260746 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.265320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1388/2802] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:25.269335 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.284689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:25.301827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:25.325728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac71049-fa5e-44d3-afbb-c7f5a84d2a8c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.343341 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8fd08-ff4a-44b0-9b92-eb76883058fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.352143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.353116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.353116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.353442 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.354032 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.356275 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.377097 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.377820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.378095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.378362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.378950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1389/2803] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:25.382904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.387936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.389390 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.392829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393512be-651c-4c70-b371-45b472984c14 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.397749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.398336 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.398607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.398873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.399109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1415/2804] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:25.399519 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.401153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.411659 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1390/2805] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:25.415707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.426514 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2210c-8121-4257-94ce-409e8d28f0b5 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.431078 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.431672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.431946 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.432234 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.432925 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.434540 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a1b0f-3a50-4385-afde-681b9a1d8632 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.444933 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1416/2806] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:25.449552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.461953 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77568-1cd5-4337-b162-6d2521b83036 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.467041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.467817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.468127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.468428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] PATH_INFO: `/v3/projects/139a01b5c3b2421aabb0edad466dbfdd/users/346da6c99f244e689a02037a0cd2599e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.485738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] RBAC: Authorizing `identity:create_grant(project_id=139a01b5c3b2421aabb0edad466dbfdd, user_id=346da6c99f244e689a02037a0cd2599e, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.488836 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.503731 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1391/2807] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:25 2022] PUT /identity/v3/projects/139a01b5c3b2421aabb0edad466dbfdd/users/346da6c99f244e689a02037a0cd2599e/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:25.575364 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.575638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.575904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.607888 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] MFA Rules not processed for user `346da6c99f244e689a02037a0cd2599e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:25.609163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.677725 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1417/2808] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:25.684513 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.686376 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.690223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.692383 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.756979 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7e6f1-4585-484d-a6af-a8c466a4cb78 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.761076 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.761694 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.761978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.762250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.771543 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.772960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd70a-ed1f-4652-b336-4628db8a698d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.773110 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.777827 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.778167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:25.778470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.778749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.779020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] PATH_INFO: `/v3/projects/34f547f2bae94d50a7ad56ba49d0f8b9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.780906 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f547f2bae94d50a7ad56ba49d0f8b9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.783027 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.786366 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1418/2809] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:25.809819 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.822436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efe5c-0a84-46ad-b70e-081ef6a573c4 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:25.827332 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:25.828027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.828368 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.828732 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] PATH_INFO: `/v3/users/346da6c99f244e689a02037a0cd2599e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.830096 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=346da6c99f244e689a02037a0cd2599e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:25.832107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:25.845618 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f547f2bae94d50a7ad56ba49d0f8b9', 'request_id': 'req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.853681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f547f2bae94d50a7ad56ba49d0f8b9', 'request_id': 'req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.860937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f547f2bae94d50a7ad56ba49d0f8b9', 'request_id': 'req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.877642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] The token cache is being invalidate because project 34f547f2bae94d50a7ad56ba49d0f8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:25.877934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.878569 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.880573 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1392/2810] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:25 2022] DELETE /identity/v3/projects/34f547f2bae94d50a7ad56ba49d0f8b9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:25.890944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346da6c99f244e689a02037a0cd2599e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.894172 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:25.896062 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:25.900052 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346da6c99f244e689a02037a0cd2599e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.910683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:25.917009 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346da6c99f244e689a02037a0cd2599e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.929293 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346da6c99f244e689a02037a0cd2599e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.930001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-cf095d6a-6321-4a35-b151-f9f30224b62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346da6c99f244e689a02037a0cd2599e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:25.942579 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1419/2811] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:25 2022] DELETE /identity/v3/users/346da6c99f244e689a02037a0cd2599e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:25.947185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:25.947635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:25.948006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:25.970928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:25.972230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.003876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee0c6a-9e77-4aa0-9ddb-e9c88ad898f0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.049077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1420/2812] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:26.061610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.064835 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.087411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.088150 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.088536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.088873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] PATH_INFO: `/v3/users/7b73d620c3b3499d8119c35b4c965a7c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.090439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b73d620c3b3499d8119c35b4c965a7c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.092663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.130607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ba553-362b-42fa-9392-e8e57a26f625 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.134552 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.135173 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.135450 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.135736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.144769 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.146077 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b73d620c3b3499d8119c35b4c965a7c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.146224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.153922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b73d620c3b3499d8119c35b4c965a7c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.154591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b73d620c3b3499d8119c35b4c965a7c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.155601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1421/2813] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:26.163343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b73d620c3b3499d8119c35b4c965a7c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.163991 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b73d620c3b3499d8119c35b4c965a7c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.183610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1393/2814] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:25 2022] DELETE /identity/v3/users/7b73d620c3b3499d8119c35b4c965a7c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:26.448167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.448920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.466989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.468922 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.473428 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8590e-9b13-480a-b6a8-f5d5e16b4f01 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.486246 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:26.519097 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:26.548247 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.548985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.549257 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.549531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] PATH_INFO: `/v3/projects/139a01b5c3b2421aabb0edad466dbfdd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.550712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=139a01b5c3b2421aabb0edad466dbfdd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.551502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c440-d17c-4a8d-aae8-92faa38af292 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.552787 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.556776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.557408 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.557759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.557938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] PATH_INFO: `/v3/projects/0ef5698e114649218291742d5f3992a4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.559080 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef5698e114649218291742d5f3992a4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.560994 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.607186 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139a01b5c3b2421aabb0edad466dbfdd', 'request_id': 'req-66208613-112f-4a72-87f5-e4b98979d0b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.616698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139a01b5c3b2421aabb0edad466dbfdd', 'request_id': 'req-66208613-112f-4a72-87f5-e4b98979d0b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.616997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef5698e114649218291742d5f3992a4', 'request_id': 'req-93e226b9-28cf-4373-a145-30c108990a7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.623221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef5698e114649218291742d5f3992a4', 'request_id': 'req-93e226b9-28cf-4373-a145-30c108990a7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.623446 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139a01b5c3b2421aabb0edad466dbfdd', 'request_id': 'req-66208613-112f-4a72-87f5-e4b98979d0b2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.630584 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef5698e114649218291742d5f3992a4', 'request_id': 'req-93e226b9-28cf-4373-a145-30c108990a7a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.635292 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] The token cache is being invalidate because project 139a01b5c3b2421aabb0edad466dbfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:26.635699 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.636890 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-66208613-112f-4a72-87f5-e4b98979d0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.640088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1422/2815] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:26 2022] DELETE /identity/v3/projects/139a01b5c3b2421aabb0edad466dbfdd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:26.645567 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb406-47a6-49d1-8c63-7c9db10e7be1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.645845 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb406-47a6-49d1-8c63-7c9db10e7be1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.646111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb406-47a6-49d1-8c63-7c9db10e7be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.647078 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] The token cache is being invalidate because project 0ef5698e114649218291742d5f3992a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:26.647373 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.648042 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.649730 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1394/2816] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:26 2022] DELETE /identity/v3/projects/0ef5698e114649218291742d5f3992a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:26.653418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.654722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.665817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cf0cb406-47a6-49d1-8c63-7c9db10e7be1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:26.666990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cf0cb406-47a6-49d1-8c63-7c9db10e7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.722196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e226b9-28cf-4373-a145-30c108990a7a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.724795 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1423/2817] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:26.726732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.727365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.727642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.727913 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] PATH_INFO: `/v3/users/70521e7c4b024fc6b563641c3418e5f8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.729085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70521e7c4b024fc6b563641c3418e5f8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.731085 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.732106 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.732412 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.732707 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.751834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:26.753074 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.761849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1424/2818] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:26.766640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.775854 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c57fd9-27ec-4156-b4c3-066a45298abe None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.781240 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.781837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.782109 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.782386 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.782948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70521e7c4b024fc6b563641c3418e5f8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.783068 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:26.783291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.784932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.789943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70521e7c4b024fc6b563641c3418e5f8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.790678 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70521e7c4b024fc6b563641c3418e5f8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.797871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1425/2819] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:26.799593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70521e7c4b024fc6b563641c3418e5f8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.800329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70521e7c4b024fc6b563641c3418e5f8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:26.802413 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.802722 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:26.804249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:26.816928 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7df1a-b531-4a7c-af15-1b99ed8493c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.821647 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1395/2820] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:26 2022] DELETE /identity/v3/users/70521e7c4b024fc6b563641c3418e5f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:26.824820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.824820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.824820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.824820 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.825231 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.825703 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:26.857037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1426/2821] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:26 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:26.861501 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:26.927581 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500950-0e9c-4ba6-84c6-c28a18813288 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:26.987435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:26.988050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:26.988329 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:26.988631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:26.989271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:26.991178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.013657 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1396/2822] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:26 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:27.017851 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.023433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:27.029632 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469ed9-9a9a-4255-8b4d-2d73b8bf2497 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.035392 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.036124 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.036520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.036868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.037577 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.039374 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.050060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1427/2823] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:27.054436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.065092 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a1358-4aaf-4c64-8f8c-a6148d932696 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.070160 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.071008 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.071554 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.071884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.072624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.074276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.086291 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1397/2824] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:27.090625 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.094401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.096458 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.102904 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6cf1c-b366-4f1c-b2e3-76806b594452 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.109169 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.110084 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.110637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.110968 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] PATH_INFO: `/v3/projects/7f379999d1194aac8ed1d0005bf9950e/users/ad91b2e748fb43dbae9f78fd3fd5b1d4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.111315 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd2d0-94db-4ec0-90c8-77d5dd662857 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.116251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.116918 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.117197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.117470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] PATH_INFO: `/v3/projects/d5e4fe22264b4d49b608d28e940ff375` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.118658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e4fe22264b4d49b608d28e940ff375)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.120697 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.140883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f379999d1194aac8ed1d0005bf9950e, user_id=ad91b2e748fb43dbae9f78fd3fd5b1d4, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.144180 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effc8c4-4261-464a-9333-99fe60e2460d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.164135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1428/2825] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:27 2022] PUT /identity/v3/projects/7f379999d1194aac8ed1d0005bf9950e/users/ad91b2e748fb43dbae9f78fd3fd5b1d4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:27.175105 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.175435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.175704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.177749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e4fe22264b4d49b608d28e940ff375', 'request_id': 'req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:27.184302 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e4fe22264b4d49b608d28e940ff375', 'request_id': 'req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:27.191654 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e4fe22264b4d49b608d28e940ff375', 'request_id': 'req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:27.193974 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:27.209682 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] MFA Rules not processed for user `ad91b2e748fb43dbae9f78fd3fd5b1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:27.212667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] The token cache is being invalidate because project d5e4fe22264b4d49b608d28e940ff375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:27.212959 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:27.213175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.213612 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99d1aa25-04c7-4129-ae7f-7ad54c84cf54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:27.215153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1398/2826] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:27 2022] DELETE /identity/v3/projects/d5e4fe22264b4d49b608d28e940ff375 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:27.221345 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c714507-8322-4875-9923-0811ed806bc4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.221876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c714507-8322-4875-9923-0811ed806bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.222149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c714507-8322-4875-9923-0811ed806bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.241732 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0c714507-8322-4875-9923-0811ed806bc4 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:27.243127 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c714507-8322-4875-9923-0811ed806bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.289688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1429/2827] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:27.303565 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.305040 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.322888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1399/2828] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:27.331136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.331423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.331698 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.350684 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:27.351886 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.360389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1400/2829] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:27.361691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef608cb0-874f-41fc-80f3-386878a93599 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.364660 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.365499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.366161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.366439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.366706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.373441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b053e3-3c96-4fb5-9b9b-fd126d9546a2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.375624 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.377107 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.378138 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.378716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.378988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.379402 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.380054 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:27.380350 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.381993 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.386781 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1430/2830] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:27.396551 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1401/2831] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:27.398309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.409387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0de674-314c-4280-a6ac-399465311e0f service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.414321 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.414918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.415195 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.415462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.416089 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.418035 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.447706 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1431/2832] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:27.452129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.463111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a13962-289b-487d-9665-41208d3a2fe1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.523157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.524090 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.524377 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.524677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.525411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.527961 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.553262 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1402/2833] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:27.558006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.569601 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47e42a-6830-4cd9-b747-0a1b7965b506 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.575414 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.576036 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.576314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.576610 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.577498 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.579152 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.592641 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1432/2834] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:27.596843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.607401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c19ac9-1f98-4f23-955f-5d966f5c4aea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.612184 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.612914 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.613194 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.613474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.614105 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.615685 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78346a4-ab90-4d8a-b46d-caa47604f3c1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.626798 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1403/2835] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:27.632055 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.647875 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cda10-ed58-4f11-b278-8f78ed587659 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.655702 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.656771 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.657277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.657789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] PATH_INFO: `/v3/projects/19c5d496144f4ecb9f101be8a5b9d513/users/0636989c6eee4bebb165be3d68b4c9ab/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.674072 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c5d496144f4ecb9f101be8a5b9d513, user_id=0636989c6eee4bebb165be3d68b4c9ab, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.676734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.692282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1433/2836] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:27 2022] PUT /identity/v3/projects/19c5d496144f4ecb9f101be8a5b9d513/users/0636989c6eee4bebb165be3d68b4c9ab/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:27.765749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.766111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.766353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.795814 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] MFA Rules not processed for user `0636989c6eee4bebb165be3d68b4c9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:27.797066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:27.838277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:27.866553 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1404/2837] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:27.878985 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:27.883673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:27.935788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a6ab0-e885-4cc5-b1c4-9fac85f5d301 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:27.939560 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:27.940161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:27.940435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:27.940728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:27.949520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:27.950998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d813e-6f5c-4583-81f0-63681c05f783 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:27.960983 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1434/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:29.448072 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:29.450681 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:29.556100 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7f78b-0837-4876-9d6d-62cb79b288bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:29.560614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:29.561399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:29.561677 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:29.561950 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] PATH_INFO: `/v3/users/1161d6181bce4c31bbb4ec8bedd2f4bc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:29.563132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1161d6181bce4c31bbb4ec8bedd2f4bc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:29.565079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:29.631937 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1161d6181bce4c31bbb4ec8bedd2f4bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:29.639171 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1161d6181bce4c31bbb4ec8bedd2f4bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:29.639828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1161d6181bce4c31bbb4ec8bedd2f4bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:29.647056 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1161d6181bce4c31bbb4ec8bedd2f4bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:29.647640 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1161d6181bce4c31bbb4ec8bedd2f4bc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:29.665204 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1405/2839] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:29 2022] DELETE /identity/v3/users/1161d6181bce4c31bbb4ec8bedd2f4bc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:29.670237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:29.670727 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:29.670825 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:29.690371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:29.691611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:29.756425 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1435/2840] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:29.766563 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:29.768264 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:29.847116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16f6ba-4c29-4710-8158-51c5ee92e401 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:29.850832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:29.851465 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:29.851743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:29.852011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:29.862576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:29.864040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6603d3-cd34-45f8-b031-c2b2853931a5 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:29.874142 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1406/2841] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:30.006371 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.007897 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.023709 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceaf69-d10a-4842-b333-179356da36d3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.030347 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.031083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.031429 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.031939 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] PATH_INFO: `/v3/users/ad91b2e748fb43dbae9f78fd3fd5b1d4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.033919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad91b2e748fb43dbae9f78fd3fd5b1d4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.037328 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.138410 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad91b2e748fb43dbae9f78fd3fd5b1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.146294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad91b2e748fb43dbae9f78fd3fd5b1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.159814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad91b2e748fb43dbae9f78fd3fd5b1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.171515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad91b2e748fb43dbae9f78fd3fd5b1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.172248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad91b2e748fb43dbae9f78fd3fd5b1d4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.179844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1436/2842] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:30 2022] DELETE /identity/v3/users/ad91b2e748fb43dbae9f78fd3fd5b1d4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:30.184753 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.185093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.185362 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.194197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:30.202291 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.203867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.208785 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:30.210215 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.227338 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:30.277025 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc719-2819-48a3-8749-fdb347ba148b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.282019 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.282650 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.282922 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.283185 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] PATH_INFO: `/v3/projects/4fef28dcd9cb43f48f082f2342419268` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.284318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fef28dcd9cb43f48f082f2342419268)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.286265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.286812 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1407/2843] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:30.297358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.299254 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.336111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fef28dcd9cb43f48f082f2342419268', 'request_id': 'req-d440bd12-bc77-4f11-8b92-a71428e9e733'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.345356 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fef28dcd9cb43f48f082f2342419268', 'request_id': 'req-d440bd12-bc77-4f11-8b92-a71428e9e733'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.351691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fef28dcd9cb43f48f082f2342419268', 'request_id': 'req-d440bd12-bc77-4f11-8b92-a71428e9e733'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.359276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9f67-cfb1-46c9-aacb-8121cef96259 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.362623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] The token cache is being invalidate because project 4fef28dcd9cb43f48f082f2342419268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:30.362905 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.363597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.365265 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1437/2844] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:30 2022] DELETE /identity/v3/projects/4fef28dcd9cb43f48f082f2342419268 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:30.369453 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.370765 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.415650 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.416280 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.416596 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.416870 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.418838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.447358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440bd12-bc77-4f11-8b92-a71428e9e733 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.457214 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.457902 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.458175 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.458431 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] PATH_INFO: `/v3/users/4e604a2190cf4d2c8c040ababfb30d23` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.459597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e604a2190cf4d2c8c040ababfb30d23)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.461635 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.487369 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.489274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.500098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1408/2845] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:30.512975 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e604a2190cf4d2c8c040ababfb30d23'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.515046 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:30.521220 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e604a2190cf4d2c8c040ababfb30d23'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.535858 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e604a2190cf4d2c8c040ababfb30d23'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.543236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e604a2190cf4d2c8c040ababfb30d23'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.543955 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e604a2190cf4d2c8c040ababfb30d23'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.550512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1438/2846] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:27:30 2022] DELETE /identity/v3/users/4e604a2190cf4d2c8c040ababfb30d23 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:30.779034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.780615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.795149 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb83b-a0b7-4653-9776-75612109b65b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.800858 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:30.848213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:30.849802 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.871586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.872242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.872604 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.872819 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] PATH_INFO: `/v3/projects/7f379999d1194aac8ed1d0005bf9950e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.874010 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f379999d1194aac8ed1d0005bf9950e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.875968 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.920301 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f976fb-c28b-4326-bdd1-25c50d42e81e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:30.923744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f379999d1194aac8ed1d0005bf9950e', 'request_id': 'req-2a94fd52-16a3-41af-80ef-ffe753264ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.929706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f379999d1194aac8ed1d0005bf9950e', 'request_id': 'req-2a94fd52-16a3-41af-80ef-ffe753264ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.935867 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f379999d1194aac8ed1d0005bf9950e', 'request_id': 'req-2a94fd52-16a3-41af-80ef-ffe753264ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.949721 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] The token cache is being invalidate because project 7f379999d1194aac8ed1d0005bf9950e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:30.950011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.950667 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2a94fd52-16a3-41af-80ef-ffe753264ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:30.952188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1409/2847] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:30 2022] DELETE /identity/v3/projects/7f379999d1194aac8ed1d0005bf9950e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:30.957161 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108822a-7f39-4288-a606-8e036738c5b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.957446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108822a-7f39-4288-a606-8e036738c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.957715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108822a-7f39-4288-a606-8e036738c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.976073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:30.976451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b108822a-7f39-4288-a606-8e036738c5b7 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:30.976759 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.977031 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.977295 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] PATH_INFO: `/v3/projects/908064553c224d46a7de5e08d65754c7` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:30.977695 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b108822a-7f39-4288-a606-8e036738c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:30.978471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908064553c224d46a7de5e08d65754c7)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:30.980361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:30.985644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1410/2848] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:30.993277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:30.993550 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:30.993815 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.013574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:31.014727 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.028490 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908064553c224d46a7de5e08d65754c7', 'request_id': 'req-1ae8b467-b21d-45e1-b4b0-468219f2e955'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:31.037380 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908064553c224d46a7de5e08d65754c7', 'request_id': 'req-1ae8b467-b21d-45e1-b4b0-468219f2e955'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:31.043844 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908064553c224d46a7de5e08d65754c7', 'request_id': 'req-1ae8b467-b21d-45e1-b4b0-468219f2e955'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:31.054728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] The token cache is being invalidate because project 908064553c224d46a7de5e08d65754c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:27:31.055016 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:31.055690 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ae8b467-b21d-45e1-b4b0-468219f2e955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:27:31.057287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1439/2849] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:27:30 2022] DELETE /identity/v3/projects/908064553c224d46a7de5e08d65754c7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:27:31.063337 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd3c8-d8ff-41da-9fe1-b3f063a5e713 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.063618 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd3c8-d8ff-41da-9fe1-b3f063a5e713 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.064237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd3c8-d8ff-41da-9fe1-b3f063a5e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.073950 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:31.075626 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1411/2850] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:31.079869 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.086249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-56afd3c8-d8ff-41da-9fe1-b3f063a5e713 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:31.087912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-56afd3c8-d8ff-41da-9fe1-b3f063a5e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.089462 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622148d-cb70-44f3-932c-9a600d5daa22 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.094155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.094692 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.094971 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.095242 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.095895 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:31.096197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.096529 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1440/2851] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:31.101591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.105944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.105944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.105944 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.118292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1412/2852] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:27:31.118446 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.129230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:31.130221 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ba96-f939-4011-969a-c1f030c72936 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.130591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.135593 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.136197 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.136479 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.136805 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.137547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.139656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.140229 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1441/2853] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:31.144352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.153892 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084e52b-faec-4316-9229-2f51a3f7ef6e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.158539 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.159165 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.159444 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.159712 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.160362 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:27:31.160695 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.162285 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.175158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1442/2854] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 28 10:27:31.178631 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:31.180359 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.189200 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1413/2855] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:31.193864 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.195233 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff62b67-9d83-4276-a315-e6b1561baaa0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.262829 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2cd4-82e6-493b-bd52-1be2efdf0ea2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.265807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.266433 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.266739 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.267403 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.267583 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.268073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.268199 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.268680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.268924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.269658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.270170 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.271859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.301904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1414/2856] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:31.303001 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1443/2857] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:31.303157 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.307125 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.315265 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9441a-c508-49bd-9c77-7c2f79acb27e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.375457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e655c0-b782-4cd5-873e-cb554d4a1a62 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.379466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.380057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.380296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.380412 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.380665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.380817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.380948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.381219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.381316 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.381883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.382903 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.383780 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.392749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1415/2858] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:31.397137 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.407597 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7938b-fc63-4715-9b44-dfdf5c9b4bf2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.409406 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1444/2859] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:31.410424 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.412371 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.412995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.413273 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.413543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.414179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.415765 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.421060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ed5c5-ff41-420b-81c2-1ca1dddfbf28 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.425817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.426430 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.426817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.426935 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1416/2860] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:31.427088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.427720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.429339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.431035 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.439686 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1445/2861] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:31.441542 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14604f-13f9-49ec-bb7e-62c7f787ced0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.443461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.446356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.447012 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.447295 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.447575 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] PATH_INFO: `/v3/projects/d82620aefc2f4df9a5aa6492c2786385/users/8e56cbeb9e4f46ffa262c788a61e4bf4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.453688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e591e38-f5d3-43fa-a59a-66ce0b8751b8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.458509 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.459161 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.459436 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.459698 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.460352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.461959 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.462915 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82620aefc2f4df9a5aa6492c2786385, user_id=8e56cbeb9e4f46ffa262c788a61e4bf4, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.465670 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7db94-a011-4382-9d14-9becc6842adc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.473327 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1446/2862] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 28 10:27:31.477504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.479509 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1417/2863] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:31 2022] PUT /identity/v3/projects/d82620aefc2f4df9a5aa6492c2786385/users/8e56cbeb9e4f46ffa262c788a61e4bf4/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:31.484223 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.484524 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.484879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.487408 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34d67b-6748-42e1-b3d0-4e4591aa83ed admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.504565 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:31.505934 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.518504 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:31.551592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.552399 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.552705 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.553045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] PATH_INFO: `/v3/projects/75a1730703ab42be820aa89accbd2bb5/users/24e130a4ca124153908eb8ca5ecf4521/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.554904 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:31.568885 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a1730703ab42be820aa89accbd2bb5, user_id=24e130a4ca124153908eb8ca5ecf4521, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.571935 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6e067-3911-4ed5-a137-0c27bf8c1fce admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.573271 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1418/2864] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:27:31.584606 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.587669 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.588592 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1447/2865] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:31 2022] PUT /identity/v3/projects/75a1730703ab42be820aa89accbd2bb5/users/24e130a4ca124153908eb8ca5ecf4521/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:31.600963 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.601250 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.601568 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.634506 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] MFA Rules not processed for user `24e130a4ca124153908eb8ca5ecf4521`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:31.635776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.651134 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c744e-094b-4e1a-982a-8736be3e5199 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.699049 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.699655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.699948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.700207 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.701733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1448/2866] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:31.714803 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:31.716205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:31.760110 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.761854 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e811-6684-4b25-a27e-1b4093242773 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.771545 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1419/2867] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:31.794919 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ec4c1-5210-4183-97c5-94d5a3a16e2e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:31.800073 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:31.800525 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:31.800821 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:31.801088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:31.809855 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:31.811300 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:31.820239 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1449/2868] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 28 10:27:36.088696 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.089225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.089225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.118943 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] MFA Rules not processed for user `8e56cbeb9e4f46ffa262c788a61e4bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:36.120176 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:36.171006 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:36.193759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1420/2869] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:36.198843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.213320 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf665-b180-4a1a-8300-d851939d0fb1 service glance] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.218743 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.219584 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.219860 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.220130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.221004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.222990 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.255470 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1450/2870] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:27:36 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:27:36.259622 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.270689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02394ed7-80e0-4f2f-95eb-7acf486a36ae None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.329178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.329855 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.330144 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.330426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.331129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.333129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.360337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1421/2871] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:27:36 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:27:36.364733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.375108 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:36.379492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f281ac0-b2f2-4aa0-a641-dfa9745d4dc3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.393793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.393793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.393793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.393793 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.394193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.394193 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.401294 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1451/2872] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:36.405831 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.416188 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb074424-89bc-44d5-8e0d-6ac77ac7f35b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.421419 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.422011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.422293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.422588 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] PATH_INFO: `/v3/projects/27ecb68fb10e4799b781d3014731462b/users/769a1669a09d4e35b8ebc7a3713a26bf/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.437122 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ecb68fb10e4799b781d3014731462b, user_id=769a1669a09d4e35b8ebc7a3713a26bf, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.439787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.456783 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1422/2873] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:36 2022] PUT /identity/v3/projects/27ecb68fb10e4799b781d3014731462b/users/769a1669a09d4e35b8ebc7a3713a26bf/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:36.469344 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.481081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21635086-0ca8-48f4-9615-585d5d6c3e43 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.542501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.543100 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.543407 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.543692 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.544339 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.546389 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd6bb4-7320-4455-91cf-e3a8f42e8a7d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.557111 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1452/2874] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:27:36.561236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:36.572256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53c358-88e3-4fc2-a8c5-e0115dc74f56 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:36.577210 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:36.577863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:36.578143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:36.578418 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] PATH_INFO: `/v3/projects/27ecb68fb10e4799b781d3014731462b/users/769a1669a09d4e35b8ebc7a3713a26bf/roles/4f6d8438189649f09ae3243d86ab3381` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:36.580578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ecb68fb10e4799b781d3014731462b, user_id=769a1669a09d4e35b8ebc7a3713a26bf, role_id=4f6d8438189649f09ae3243d86ab3381)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:36.583307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:36.593862 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1423/2875] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:27:36 2022] PUT /identity/v3/projects/27ecb68fb10e4799b781d3014731462b/users/769a1669a09d4e35b8ebc7a3713a26bf/roles/4f6d8438189649f09ae3243d86ab3381 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:27:41.205249 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bbf8-3cfa-438d-bcf3-99c4e7527ad6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:41.205839 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bbf8-3cfa-438d-bcf3-99c4e7527ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:41.205936 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bbf8-3cfa-438d-bcf3-99c4e7527ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:41.241814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8a14bbf8-3cfa-438d-bcf3-99c4e7527ad6 None None] MFA Rules not processed for user `769a1669a09d4e35b8ebc7a3713a26bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:41.243325 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8a14bbf8-3cfa-438d-bcf3-99c4e7527ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:41.330045 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1453/2876] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:41.342808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:41.407422 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae394d-3d4f-4e57-994c-b5a13329d0b8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:41.411673 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fb55fa700442dd974594ada94e7bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:41.412401 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:41.412772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:41.413222 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:41.467040 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:41.476607 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:41.478653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:41.489066 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1424/2877] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:41.613037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373125-b0ad-4e7f-afee-3badf108c2a4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:41.613719 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373125-b0ad-4e7f-afee-3badf108c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:41.614088 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373125-b0ad-4e7f-afee-3badf108c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:41.692953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-03373125-b0ad-4e7f-afee-3badf108c2a4 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:41.694171 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-03373125-b0ad-4e7f-afee-3badf108c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:41.745623 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:27:41.766863 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1454/2878] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:27:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:41.817502 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:41.819000 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:41.926441 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea237-0ff6-4c95-9981-97610c43324d service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:41.936033 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:41.936728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:41.937015 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:41.937428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:41.947578 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:41.950187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:41.959358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1425/2879] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:27:50.756516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:50.757310 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:50.757646 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:50.777306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] MFA Rules not processed for user `8e56cbeb9e4f46ffa262c788a61e4bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:50.778513 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:50.785369 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1455/2880] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:51.055598 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:51.057179 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:51.117821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398aa-2fd2-4c25-9713-19bfd1f15068 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:51.122439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:51.123034 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:51.123312 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:51.123582 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:51.125676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:51.136147 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:51.137627 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c603d-39dc-425e-9a28-159db53e057f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:51.146038 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1426/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:51.474948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:51.479838 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:51.577048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e252-ae28-45e5-a85c-e22fc2baac7e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:51.581952 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:51.582566 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:51.582842 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:51.583117 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:51.584887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:51.596534 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:51.598213 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:51.608287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1456/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:27:53.208214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:53.208857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:53.208857 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:53.227966 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:27:53.229206 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:27:53.236610 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1427/2883] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:27:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:27:53.249353 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:27:53.261689 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42894b79-cc05-45c4-a28b-b32c13a7b8e9 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:27:53.267378 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:27:53.267984 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:27:53.268258 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:27:53.268546 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:27:53.276830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:27:53.278415 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:27:53.286590 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1457/2884] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:27:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:28:03.222866 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=74 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:03.283316 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=47 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:12.377435 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:12.379164 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:12.459766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a5b6b-5436-4d78-a98b-64a097759357 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:12.465263 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:12.465875 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:12.466153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:12.466428 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:12.468169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:12.481116 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:12.482797 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:12.496544 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1428/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:14.712806 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:14.714532 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:14.774334 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f6590-6f52-4793-87bb-5c04c6384c3d service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:14.778499 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:14.779095 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:14.779373 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:14.779640 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:14.781531 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:14.841007 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:14.842438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a35aa-e025-435b-8a9d-bfb49e5a9394 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:14.851809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1458/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:22.490318 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=34 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:23.378715 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:23.390652 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe319-7f3f-434a-adca-c5993c9001b7 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:23.396356 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:23.396989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:23.397270 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:23.397543 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:23.399347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:23.412605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:23.414219 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:23.425984 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1429/2887] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:24.847537 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=48 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:29.125004 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c4f11-0610-46ad-97ba-c18de05c9119 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:29.125680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c4f11-0610-46ad-97ba-c18de05c9119 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:29.125680 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c4f11-0610-46ad-97ba-c18de05c9119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:29.145796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b91c4f11-0610-46ad-97ba-c18de05c9119 None None] MFA Rules not processed for user `3deb7adb33004f1097574470754611ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:29.147224 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b91c4f11-0610-46ad-97ba-c18de05c9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:29.154846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1459/2888] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:28:33.421123 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:39.139837 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:42.801249 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:42.864201 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76546d-6984-4e3a-9996-2229e70b5c2f service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:42.868898 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:42.869533 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:42.869802 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:42.870079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] PATH_INFO: `/v3/users/24e130a4ca124153908eb8ca5ecf4521` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:42.871214 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e130a4ca124153908eb8ca5ecf4521)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:42.873198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:42.932256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e130a4ca124153908eb8ca5ecf4521'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:42.942615 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e130a4ca124153908eb8ca5ecf4521'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:42.943552 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e130a4ca124153908eb8ca5ecf4521'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:42.953888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e130a4ca124153908eb8ca5ecf4521'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:42.954559 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e130a4ca124153908eb8ca5ecf4521'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:42.972705 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1430/2889] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:28:42 2022] DELETE /identity/v3/users/24e130a4ca124153908eb8ca5ecf4521 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:28:42.977455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:42.977949 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:42.978086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:42.997627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:42.998871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.060048 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1460/2890] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:28:43.071484 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.073224 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.141124 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eb91d-3ac5-4018-998d-0a5580c09146 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.145145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.145769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.146076 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.146351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.156011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.157516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.167241 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1431/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:43.441597 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.443057 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.455971 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afc31-d27c-436e-b850-d10dc9eb3543 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.461865 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.462987 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.463616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.463826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] PATH_INFO: `/v3/projects/75a1730703ab42be820aa89accbd2bb5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.465452 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a1730703ab42be820aa89accbd2bb5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.467830 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.482167 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.483877 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.498473 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8c250-60a4-4235-94eb-a5ed89bf3859 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.503274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.503907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.504186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.504455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] PATH_INFO: `/v3/users/0636989c6eee4bebb165be3d68b4c9ab` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.505614 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0636989c6eee4bebb165be3d68b4c9ab)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.507562 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.521688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a1730703ab42be820aa89accbd2bb5', 'request_id': 'req-c3cef5be-a154-4d6a-a252-0427b982552e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.526941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:43.532365 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a1730703ab42be820aa89accbd2bb5', 'request_id': 'req-c3cef5be-a154-4d6a-a252-0427b982552e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.540091 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a1730703ab42be820aa89accbd2bb5', 'request_id': 'req-c3cef5be-a154-4d6a-a252-0427b982552e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.555553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] The token cache is being invalidate because project 75a1730703ab42be820aa89accbd2bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:28:43.555953 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.556809 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c3cef5be-a154-4d6a-a252-0427b982552e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.558358 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1461/2892] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:28:43 2022] DELETE /identity/v3/projects/75a1730703ab42be820aa89accbd2bb5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:28:43.563999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85952b3-2c7f-44d1-8dbc-e7c3fa4498a5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.564289 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85952b3-2c7f-44d1-8dbc-e7c3fa4498a5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.564591 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85952b3-2c7f-44d1-8dbc-e7c3fa4498a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.573769 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0636989c6eee4bebb165be3d68b4c9ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.581658 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0636989c6eee4bebb165be3d68b4c9ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.582414 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0636989c6eee4bebb165be3d68b4c9ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.585672 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e85952b3-2c7f-44d1-8dbc-e7c3fa4498a5 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:43.586912 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e85952b3-2c7f-44d1-8dbc-e7c3fa4498a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.592879 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0636989c6eee4bebb165be3d68b4c9ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.593708 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-508ab5da-8ee1-479d-81be-268ad8b7035c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0636989c6eee4bebb165be3d68b4c9ab'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:43.632644 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1432/2893] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:28:43 2022] DELETE /identity/v3/users/0636989c6eee4bebb165be3d68b4c9ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:28:43.637397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.637967 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.638874 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.661111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:43.662437 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.679405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1462/2894] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:28:43.687087 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.687375 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.687647 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.711135 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:43.712379 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.731876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1433/2895] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:28:43.742320 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.743884 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:43.748496 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:43.793468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1463/2896] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:28:43.798060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.807318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcecca-c5f6-41ed-9f31-2b0a769da3c1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.808711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762fcdc-36a9-4efd-9e1a-698b8d4af344 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.811980 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.812547 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.812688 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.812878 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.813146 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.813256 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.813447 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.813680 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.813796 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 28 10:28:43.814077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.815673 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.822828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.824257 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.830558 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1464/2897] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Aug 28 10:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 28 10:28:43.832081 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.843653 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1434/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:43.844931 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5e0d6-abd8-4777-a3c6-c09dd4d30df5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.850405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.851018 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.851299 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.851570 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.852205 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.854189 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.890502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1465/2899] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Aug 28 10:28:43 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 28 10:28:43.891706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:43.901439 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e685-f89c-47f8-901f-9327604f1416 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:43.939131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:43.966787 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:43.967560 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:43.967844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:43.968145 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:43.968828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:43.970782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:43.996849 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1435/2900] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Aug 28 10:28:43 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 28 10:28:43.998130 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:44.008910 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a828-6b77-4dfd-be87-518458aaf7ff admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:44.014510 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:44.015126 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.015468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.015663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.016306 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:44.017948 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:44.029607 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1466/2901] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:28:44.034196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:44.045287 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c884098-6a91-4d48-bba8-b73d04447d23 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:44.050129 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:44.050745 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.051024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.051301 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.051948 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:44.053574 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45d4b5-5ef8-42bc-9023-64d434eb7763 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:44.066808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1436/2902] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Aug 28 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 28 10:28:44.075714 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:44.087747 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08eb15-26e8-4e96-bb06-56dc85104dbd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:44.092608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:44.093269 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.093547 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.093816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] PATH_INFO: `/v3/projects/d011dbdfa9074d418d341f1d23991844/users/c3c27f3d5edb4eb08546ccda853de534/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.109252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d011dbdfa9074d418d341f1d23991844, user_id=c3c27f3d5edb4eb08546ccda853de534, role_id=5bd8b3d7d5524f00bbdf86a2ccf0a6c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:44.112034 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:44.127179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1467/2903] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Aug 28 10:28:44 2022] PUT /identity/v3/projects/d011dbdfa9074d418d341f1d23991844/users/c3c27f3d5edb4eb08546ccda853de534/roles/5bd8b3d7d5524f00bbdf86a2ccf0a6c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:28:44.137572 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.137897 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.138163 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.167570 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] MFA Rules not processed for user `c3c27f3d5edb4eb08546ccda853de534`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:44.168871 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:44.213581 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:44.215639 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:44.245989 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1437/2904] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:28:44.257119 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:44.258995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:44.297976 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dbd42-3156-4b42-98ac-af3e5c796ee1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:44.303704 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:44.304354 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.304676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.304954 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] PATH_INFO: `/v3/projects/19c5d496144f4ecb9f101be8a5b9d513` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.306208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c5d496144f4ecb9f101be8a5b9d513)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:44.309150 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:44.322455 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d41bc-8a8a-4ea8-8487-6612d01cc5a5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:44.326841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77f82ddcd0604a3697dc8b571719e1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:44.327683 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:44.328131 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:44.328409 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:44.338196 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:44.339666 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:44.349444 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1438/2905] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:44.367198 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c5d496144f4ecb9f101be8a5b9d513', 'request_id': 'req-e0443206-d7c8-49a7-b45c-6f466a5ee926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:44.378098 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c5d496144f4ecb9f101be8a5b9d513', 'request_id': 'req-e0443206-d7c8-49a7-b45c-6f466a5ee926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:44.379691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:44.387237 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c5d496144f4ecb9f101be8a5b9d513', 'request_id': 'req-e0443206-d7c8-49a7-b45c-6f466a5ee926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:44.401435 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] The token cache is being invalidate because project 19c5d496144f4ecb9f101be8a5b9d513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:28:44.401826 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:44.402608 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0443206-d7c8-49a7-b45c-6f466a5ee926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:28:44.404391 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1468/2906] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:28:44 2022] DELETE /identity/v3/projects/19c5d496144f4ecb9f101be8a5b9d513 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:28:45.023308 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:45.078992 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:28:45.083863 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b554-88df-4b08-bc1b-6eb6aaa8ca56 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:45.135384 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:45.136047 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:45.136332 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:45.136655 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:45.138605 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:45.208782 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:45.210397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:45.220841 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1439/2907] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:45.760602 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:45.761159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:45.761159 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:45.780807 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] MFA Rules not processed for user `c3c27f3d5edb4eb08546ccda853de534`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:28:45.782020 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:45.837457 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1469/2908] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:28:49.984358 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:28:49.988483 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:50.091988 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6497d3-6fe9-45df-a954-09776b9949ef service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:28:50.099907 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbe80d5de4bb4746a1756d1077e647a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:28:50.100641 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:28:50.100924 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:28:50.101198 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:28:50.103060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:28:50.119407 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:28:50.122063 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91055f19-e907-400a-9081-8a3db66c4931 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:28:50.137168 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1440/2909] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:28:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:28:55.830813 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=30,keystone:DELETE=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:29:00.130856 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:29:02.361123 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:02.362611 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:02.374972 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcd22c-641b-4675-926f-3ff50b5c302f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:02.378631 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:02.379244 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:02.379517 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:02.379782 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:02.381512 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:02.446275 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:02.447737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:02.457722 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1470/2910] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:29:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:29:04.102892 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9aad4-57e1-4f94-99f8-0bfbc0f6f992 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:04.103516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9aad4-57e1-4f94-99f8-0bfbc0f6f992 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:04.103516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9aad4-57e1-4f94-99f8-0bfbc0f6f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:04.131191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e8e9aad4-57e1-4f94-99f8-0bfbc0f6f992 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:29:04.132496 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8e9aad4-57e1-4f94-99f8-0bfbc0f6f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:04.220014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1441/2911] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:29:04.282817 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:04.284277 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:04.369850 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e30b9-6b4c-4bfc-bf96-60fc9e6e1b8c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:04.376305 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:04.377248 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:04.377522 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:04.377786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:04.390492 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:04.394411 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:04.408187 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1471/2912] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:29:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:29:05.011424 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d450d-d54b-4af8-9fdc-9b0242c135ce None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.011711 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d450d-d54b-4af8-9fdc-9b0242c135ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.011982 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d450d-d54b-4af8-9fdc-9b0242c135ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.031817 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c32d450d-d54b-4af8-9fdc-9b0242c135ce None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:29:05.033098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c32d450d-d54b-4af8-9fdc-9b0242c135ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:05.041136 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1442/2913] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:29:05.053834 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:05.064773 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b1d0-1720-4239-ae57-10610f8a1a76 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:05.069792 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:05.070398 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.070788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.071060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.080083 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:05.081731 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:05.090833 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1472/2914] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:29:05.328397 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9abbc5-e6a5-4b92-8a7e-512c270e5f17 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.328894 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9abbc5-e6a5-4b92-8a7e-512c270e5f17 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.329004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9abbc5-e6a5-4b92-8a7e-512c270e5f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.348391 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-af9abbc5-e6a5-4b92-8a7e-512c270e5f17 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:29:05.349638 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-af9abbc5-e6a5-4b92-8a7e-512c270e5f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:05.358148 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1443/2915] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Aug 28 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:29:05.370219 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:05.381183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d41034-b8db-4f91-83c6-17586c3356ee service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:05.389520 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:05.390121 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.390434 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.390728 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.400661 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:05.402309 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:05.410730 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1473/2916] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:29:05.741675 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.741960 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.742225 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.762490 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:29:05.763691 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:05.772155 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1444/2917] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:29:05.784228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:05.795788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df36215-cde9-4cc3-9331-ebfb3da970c7 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:05.802014 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:05.802620 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:05.802900 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:05.803172 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:05.811281 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:05.812973 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142111f-4def-426d-a44f-3ac0d0777829 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:05.821330 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1474/2918] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:29:07.978586 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:07.980159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:08.049050 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21d406-b788-4d80-b601-265eca832e55 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:08.054451 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:08.055079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:08.055337 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:08.055624 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] PATH_INFO: `/v3/users/c3c27f3d5edb4eb08546ccda853de534` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:08.056921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c27f3d5edb4eb08546ccda853de534)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:08.058843 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:08.109672 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c27f3d5edb4eb08546ccda853de534'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.117169 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c27f3d5edb4eb08546ccda853de534'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.117809 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c27f3d5edb4eb08546ccda853de534'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.127159 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c27f3d5edb4eb08546ccda853de534'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.127792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c27f3d5edb4eb08546ccda853de534'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.145767 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1445/2919] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:29:07 2022] DELETE /identity/v3/users/c3c27f3d5edb4eb08546ccda853de534 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:29:08.150384 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:08.150893 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:08.151006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:08.170734 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] MFA Rules not processed for user `9a1bcfcdf46a4813b52d81367ec40dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:29:08.171989 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:08.215284 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:29:08.234438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1475/2920] 10.222.0.40 () {58 vars in 958 bytes} [Sun Aug 28 10:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 28 10:29:08.244743 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:08.246528 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:08.254766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:29:08.308296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce59bd-ea06-4809-bc1b-51666117ecb4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:08.314844 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:08.315772 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:08.316141 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:08.316487 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:08.329834 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:08.331274 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:08.341011 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1446/2921] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:29:08.532179 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:29:08.533846 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:29:08.546464 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c628124-63ee-4de4-b6f2-6dd518d87d6f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:29:08.551253 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:29:08.551883 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:29:08.552167 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:29:08.552438 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] PATH_INFO: `/v3/projects/d011dbdfa9074d418d341f1d23991844` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:29:08.554077 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d011dbdfa9074d418d341f1d23991844)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:29:08.555589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:29:08.605361 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd011dbdfa9074d418d341f1d23991844', 'request_id': 'req-7bc81f86-8c64-4936-bdbb-49dfea643856'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.614050 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd011dbdfa9074d418d341f1d23991844', 'request_id': 'req-7bc81f86-8c64-4936-bdbb-49dfea643856'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.620774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd011dbdfa9074d418d341f1d23991844', 'request_id': 'req-7bc81f86-8c64-4936-bdbb-49dfea643856'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.631655 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] The token cache is being invalidate because project d011dbdfa9074d418d341f1d23991844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:29:08.631945 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.632658 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:29:08.634263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1476/2922] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:29:08 2022] DELETE /identity/v3/projects/d011dbdfa9074d418d341f1d23991844 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:29:18.337051 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=24 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:29:18.628627 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:30:09.926964 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:30:09.928426 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:30:09.999734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59406b3-59be-4222-8bb1-cd9327a70b79 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:30:10.063255 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:30:10.063873 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:30:10.064153 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:30:10.064421 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:30:10.066297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:30:10.077944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:30:10.079536 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:30:10.089756 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1447/2923] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:30:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:30:18.742385 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:30:18.743864 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:30:18.801670 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81f86-8c64-4936-bdbb-49dfea643856 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:30:18.805183 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:30:18.805783 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:30:18.806059 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:30:18.806324 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:30:18.808176 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:30:18.821760 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:30:18.823208 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:30:18.832086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1477/2924] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:30:18.938508 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:30:18.948133 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738ac5-fec8-4e34-a591-ee9e8ad6f905 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:30:18.952060 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:30:18.952701 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:30:18.952980 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:30:18.953251 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:30:18.955014 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:30:19.027175 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:30:19.028541 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:30:19.039995 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1448/2925] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:30:28.828263 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:30:29.035037 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:03.179798 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:03.189426 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830ee8-7749-415b-ac18-bbf555f0cbcb service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:03.194060 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:03.194691 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:03.194920 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:03.195228 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:03.197377 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:03.211755 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:03.213236 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451532-a759-40e7-a360-11dec13b52e8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:03.224788 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1478/2926] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:31:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:31:05.821292 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:05.831004 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68060ee-361e-48db-9da9-8693b43f0931 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:05.835226 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:05.836349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:05.836676 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:05.836941 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:05.838885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:05.854020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:05.855642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:05.866859 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1449/2927] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:31:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:31:13.219737 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:14.416268 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:14.416868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:14.416868 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:14.437816 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] MFA Rules not processed for user `207eced18c9b455186567c94927a9754`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 28 10:31:14.439027 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:14.493163 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1479/2928] 10.222.0.40 () {58 vars in 967 bytes} [Sun Aug 28 10:31:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 28 10:31:14.539399 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:14.540931 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:14.614108 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d0b8e-736d-44ce-be2b-f23985d4533f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:14.619343 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9ccc207322482e818b7c9f49aeed29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:14.619955 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:14.620349 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:14.620662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:14.629107 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:14.630759 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:14.638997 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1450/2929] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Aug 28 10:31:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 28 10:31:18.389314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:18.390867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:18.405776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1833-3708-41ff-b359-77b30b8afdc3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:18.410906 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:18.411505 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:18.411776 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:18.412039 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:18.413814 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:18.426002 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:18.427609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:18.438462 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1480/2930] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:31:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:31:21.048591 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:21.050079 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:21.062239 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322d2e7-5878-4009-8593-a6b8f96b276a service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:21.065956 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:21.066555 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:21.066828 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:21.067143 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:21.068978 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:21.134135 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:21.135576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:21.146365 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1451/2931] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:31:22.073736 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:22.092348 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da78d2-162e-4791-9db2-7639c9ee59fa service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:22.109314 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:22.112914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:22.112914 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:22.118461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:22.131174 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:22.156619 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:22.158422 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:22.179956 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1481/2932] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:31:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:31:22.192069 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:22.193774 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:22.225939 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fe4b8-c2ca-419d-9675-d912a524c54a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:22.233321 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:22.233986 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:22.234267 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:22.234539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] PATH_INFO: `/v3/users/8e56cbeb9e4f46ffa262c788a61e4bf4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:22.235706 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e56cbeb9e4f46ffa262c788a61e4bf4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:22.238353 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:22.293599 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e56cbeb9e4f46ffa262c788a61e4bf4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.302478 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e56cbeb9e4f46ffa262c788a61e4bf4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.303229 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e56cbeb9e4f46ffa262c788a61e4bf4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.316132 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e56cbeb9e4f46ffa262c788a61e4bf4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.316799 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e56cbeb9e4f46ffa262c788a61e4bf4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.319779 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:22.339981 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1452/2933] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:31:22 2022] DELETE /identity/v3/users/8e56cbeb9e4f46ffa262c788a61e4bf4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:22.515801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:22.527261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab172d-069a-49ff-b83e-b6d02254b21b service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:22.596204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:22.596882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:22.597153 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:22.597417 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] PATH_INFO: `/v3/projects/d82620aefc2f4df9a5aa6492c2786385` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:22.598524 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82620aefc2f4df9a5aa6492c2786385)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:22.600448 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:22.654307 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82620aefc2f4df9a5aa6492c2786385', 'request_id': 'req-7d4759ee-6297-4d36-8974-de02f7902bb6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.665882 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82620aefc2f4df9a5aa6492c2786385', 'request_id': 'req-7d4759ee-6297-4d36-8974-de02f7902bb6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.673445 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82620aefc2f4df9a5aa6492c2786385', 'request_id': 'req-7d4759ee-6297-4d36-8974-de02f7902bb6'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.685405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] The token cache is being invalidate because project d82620aefc2f4df9a5aa6492c2786385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:31:22.685923 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.686853 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.688927 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1482/2934] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:31:22 2022] DELETE /identity/v3/projects/d82620aefc2f4df9a5aa6492c2786385 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:22.692776 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:22.694296 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:22.708307 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed75177-14cd-45d0-9218-1261de5ae42e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:22.775716 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:22.776485 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:22.776876 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:22.777299 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] PATH_INFO: `/v3/users/769a1669a09d4e35b8ebc7a3713a26bf` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:22.778663 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=769a1669a09d4e35b8ebc7a3713a26bf)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:22.780738 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:22.838382 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769a1669a09d4e35b8ebc7a3713a26bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.847493 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769a1669a09d4e35b8ebc7a3713a26bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.848244 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769a1669a09d4e35b8ebc7a3713a26bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.857642 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769a1669a09d4e35b8ebc7a3713a26bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.858491 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769a1669a09d4e35b8ebc7a3713a26bf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:22.882351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1453/2935] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:31:22 2022] DELETE /identity/v3/users/769a1669a09d4e35b8ebc7a3713a26bf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:23.058026 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:23.059724 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:23.085447 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4759ee-6297-4d36-8974-de02f7902bb6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:23.094134 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:23.167562 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:23.168204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:23.168485 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:23.168789 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] PATH_INFO: `/v3/projects/27ecb68fb10e4799b781d3014731462b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:23.170282 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ecb68fb10e4799b781d3014731462b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:23.172223 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:23.228663 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ecb68fb10e4799b781d3014731462b', 'request_id': 'req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:23.236998 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ecb68fb10e4799b781d3014731462b', 'request_id': 'req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:23.243616 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ecb68fb10e4799b781d3014731462b', 'request_id': 'req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:23.254716 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] The token cache is being invalidate because project 27ecb68fb10e4799b781d3014731462b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:31:23.255230 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:23.255884 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:23.257439 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1483/2936] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:31:23 2022] DELETE /identity/v3/projects/27ecb68fb10e4799b781d3014731462b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:27.444178 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:27.445686 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:27.467662 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0798f-6e67-493d-9223-c411993f82fc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:27.522041 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:27.522665 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:27.522938 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:27.523192 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] PATH_INFO: `/v3/users/3deb7adb33004f1097574470754611ec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:27.524351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3deb7adb33004f1097574470754611ec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:27.526288 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:27.528656 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:27.575521 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3deb7adb33004f1097574470754611ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.582975 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3deb7adb33004f1097574470754611ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.583734 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3deb7adb33004f1097574470754611ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.595466 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3deb7adb33004f1097574470754611ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.596277 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3deb7adb33004f1097574470754611ec'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.614689 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1454/2937] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:31:27 2022] DELETE /identity/v3/users/3deb7adb33004f1097574470754611ec => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:27.799774 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:27.801468 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:27.814733 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc6c5d-af0a-4a2e-b14e-cf321959649b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:27.873287 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:27.873947 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:27.874209 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:27.874657 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] PATH_INFO: `/v3/projects/af70fccdef544a499725509ccd8c545f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:27.875778 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af70fccdef544a499725509ccd8c545f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:27.877786 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:27.925270 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:27.933423 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af70fccdef544a499725509ccd8c545f', 'request_id': 'req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.943318 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af70fccdef544a499725509ccd8c545f', 'request_id': 'req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.949676 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af70fccdef544a499725509ccd8c545f', 'request_id': 'req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.961215 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] The token cache is being invalidate because project af70fccdef544a499725509ccd8c545f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:31:27.961501 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.962261 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:27.963862 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1484/2938] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:31:27 2022] DELETE /identity/v3/projects/af70fccdef544a499725509ccd8c545f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:31:27.968247 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:27.969766 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:27.983281 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb9ef-63ae-4d35-8a73-dd34e9f7285b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:28.040539 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:28.041191 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:28.041470 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:28.041742 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] PATH_INFO: `/v3/users/9ef733aae36e477b835993ef48e7ca1f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:28.042885 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef733aae36e477b835993ef48e7ca1f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:28.045187 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:28.095260 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef733aae36e477b835993ef48e7ca1f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.101792 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef733aae36e477b835993ef48e7ca1f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.102450 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef733aae36e477b835993ef48e7ca1f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.111423 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef733aae36e477b835993ef48e7ca1f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.112117 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef733aae36e477b835993ef48e7ca1f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.125098 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:31:28.129477 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1455/2939] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:31:27 2022] DELETE /identity/v3/users/9ef733aae36e477b835993ef48e7ca1f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:28.306636 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:31:28.308194 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:31:28.324372 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e223b-05b4-497d-aa74-9b8d40b52ec8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:31:28.385871 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:31:28.386818 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:31:28.387086 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:31:28.387352 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] PATH_INFO: `/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:31:28.388516 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f46e3e7c6a41fda0f140b80e320a0c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:31:28.390472 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:31:28.448638 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f46e3e7c6a41fda0f140b80e320a0c', 'request_id': 'req-a2d75c00-b592-4150-b64d-ac6b34c9d681'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.458824 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f46e3e7c6a41fda0f140b80e320a0c', 'request_id': 'req-a2d75c00-b592-4150-b64d-ac6b34c9d681'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.466204 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f46e3e7c6a41fda0f140b80e320a0c', 'request_id': 'req-a2d75c00-b592-4150-b64d-ac6b34c9d681'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.479484 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] The token cache is being invalidate because project 96f46e3e7c6a41fda0f140b80e320a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:31:28.479794 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.480556 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:31:28.482232 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1485/2940] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:31:28 2022] DELETE /identity/v3/projects/96f46e3e7c6a41fda0f140b80e320a0c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:31:38.476405 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:32:13.070888 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:13.073089 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:13.135389 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a07f2-ba94-4fa5-bb7a-5feccbbfe5cb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:13.186234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:13.186832 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:13.187102 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:13.187375 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:13.189278 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:13.255749 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:13.257236 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:13.266576 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1456/2941] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:32:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:32:14.479164 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:14.481967 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:14.508006 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d75c00-b592-4150-b64d-ac6b34c9d681 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:14.512011 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:14.512668 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:14.512978 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:14.513252 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:14.515197 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:14.525867 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:14.527589 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:14.538154 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1486/2942] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:32:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:32:18.318434 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:18.327366 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678d8e-61fd-4273-a042-f2bbe0085d44 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:18.331233 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f71b5efa95b41e196a505798527904d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:18.331821 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:18.332094 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:18.332361 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:18.334234 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:18.397944 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:18.399474 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:18.409331 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1457/2943] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:32:24.444409 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:24.446162 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:24.462037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97fc3b-c638-477b-84fc-b71e6d077d4d service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:24.466609 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:24.467238 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:24.467515 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:24.467784 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] PATH_INFO: `/v3/users/ec6cbf7f279a4b51994ec4508be71668` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:24.468962 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6cbf7f279a4b51994ec4508be71668)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:24.470887 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:24.521493 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6cbf7f279a4b51994ec4508be71668'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.528497 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6cbf7f279a4b51994ec4508be71668'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.541158 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6cbf7f279a4b51994ec4508be71668'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.552751 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6cbf7f279a4b51994ec4508be71668'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.553461 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6cbf7f279a4b51994ec4508be71668'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.559801 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1487/2944] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:32:24 2022] DELETE /identity/v3/users/ec6cbf7f279a4b51994ec4508be71668 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:32:24.746942 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:24.753276 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:32:24.759516 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054a53-a050-42f0-a748-2981548c858d service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:24.818111 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:24.818744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:24.819020 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:24.819290 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] PATH_INFO: `/v3/projects/805bd1eda126437d9c2997927d9da8c3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:24.820411 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805bd1eda126437d9c2997927d9da8c3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:24.822367 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:24.870482 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805bd1eda126437d9c2997927d9da8c3', 'request_id': 'req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.876808 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805bd1eda126437d9c2997927d9da8c3', 'request_id': 'req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.883936 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805bd1eda126437d9c2997927d9da8c3', 'request_id': 'req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.898838 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] The token cache is being invalidate because project 805bd1eda126437d9c2997927d9da8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:32:24.899151 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.899771 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:24.901347 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1458/2945] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:32:24 2022] DELETE /identity/v3/projects/805bd1eda126437d9c2997927d9da8c3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:32:24.905471 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:24.906964 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:24.921127 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07c3f9-2da4-4c63-93d8-59551435ddc5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:24.979553 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:24.980181 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:24.980455 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:24.980758 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] PATH_INFO: `/v3/users/888e69fe4f9d4f7291c275382b918460` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:24.981918 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888e69fe4f9d4f7291c275382b918460)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:24.983843 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:25.033140 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888e69fe4f9d4f7291c275382b918460'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.039849 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888e69fe4f9d4f7291c275382b918460'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.044122 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:32:25.053502 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888e69fe4f9d4f7291c275382b918460'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.063037 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888e69fe4f9d4f7291c275382b918460'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.063711 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888e69fe4f9d4f7291c275382b918460'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.070082 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1488/2946] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Aug 28 10:32:24 2022] DELETE /identity/v3/users/888e69fe4f9d4f7291c275382b918460 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 28 10:32:25.253186 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:32:25.254717 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:32:25.270728 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba04a8-c9db-45c0-940e-32f5366fe2b3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:32:25.332093 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1bcfcdf46a4813b52d81367ec40dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371955716dbe4a589ec9dad82a9828ca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:32:25.332744 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:32:25.333024 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:32:25.333293 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] PATH_INFO: `/v3/projects/1a9fe9034993482faa627a742fecc63f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:32:25.334489 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9fe9034993482faa627a742fecc63f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:32:25.336395 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:32:25.379917 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:32:25.390297 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9fe9034993482faa627a742fecc63f', 'request_id': 'req-164bdf29-5eec-44c7-8551-d1c32f6efb77'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.395921 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9fe9034993482faa627a742fecc63f', 'request_id': 'req-164bdf29-5eec-44c7-8551-d1c32f6efb77'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.402436 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9fe9034993482faa627a742fecc63f', 'request_id': 'req-164bdf29-5eec-44c7-8551-d1c32f6efb77'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.417351 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] The token cache is being invalidate because project 1a9fe9034993482faa627a742fecc63f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 28 10:32:25.417702 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.418294 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-164bdf29-5eec-44c7-8551-d1c32f6efb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 28 10:32:25.420190 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1459/2947] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Aug 28 10:32:25 2022] DELETE /identity/v3/projects/1a9fe9034993482faa627a742fecc63f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 28 10:32:35.066720 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:32:35.414209 dv-cinder-852009-4-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 10:35:15.854093 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 28 10:35:15.856112 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:35:16.005999 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d97126-84d6-4843-be8c-eefae29e2286 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 28 10:35:16.066030 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca82b32794ef4e37b9c2608a6e78930f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f7fa10d292c4804a7550be0196a1b9f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 28 10:35:16.066637 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 28 10:35:16.066932 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 28 10:35:16.067200 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 28 10:35:16.069066 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 28 10:35:16.080387 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 28 10:35:16.082017 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd71cc6-dc3d-4b20-afcc-fcd7f1992e02 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 28 10:35:16.092491 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1489/2948] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Aug 28 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 28 10:35:26.087886 dv-cinder-852009-4-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}